[plum] Tutorial: Sized Structure Member¶
This tutorial shows how to use the sized_member() function within a structure
definition to associate a size member and another variably sized member to one another.
During unpack operations, the association causes the the unpacked size member to
control the number of bytes that are available during the unpacking of the associated
member. The association allows the size member to be left unspecified and the value of
it derived from the other member.
The examples shown on this page require the following setup:
>>> from plum.array import ArrayX
>>> from plum.bigendian import uint8
>>> from plum.structure import member, sized_member, Structure
Basics¶
To demonstrate, examine the example below. The example uses an undimensioned
array transform since it acts greedy (it varies in size and consumes all bytes
given to it). If left to unpack as normal, the array member consumes every byte
including the byte intended for the bookmark. The use of the sized_member()
function associates the array member with the size member. This
causes the unpacking operation to only allow the array member to consume the
number of bytes specified by the size member.
>>> array = ArrayX(fmt=uint8) # greedy array
>>>
>>> class Struct1(Structure):
... size: int = member(fmt=uint8, compute=True)
... array: list = sized_member(size=size, fmt=array)
... bookend: int = member(fmt=uint8)
...
>>> Struct1.unpack(b'\x02\x01\x02\x99').dump()
+--------+---------+-------+-------+---------------------+
| Offset | Access | Value | Bytes | Format |
+--------+---------+-------+-------+---------------------+
| | | | | Struct1 (Structure) |
| 0 | size | 2 | 02 | uint8 |
| | array | | | List[int] |
| 1 | [0] | 1 | 01 | uint8 |
| 2 | [1] | 2 | 02 | uint8 |
| 3 | bookend | 153 | 99 | uint8 |
+--------+---------+-------+-------+---------------------+
This association also the size to be left unspecified
when instantiating the structure with member values:
>>> Struct1(array=[1, 2], bookend=0x99).dump()
+--------+---------+-------+-------+---------------------+
| Offset | Access | Value | Bytes | Format |
+--------+---------+-------+-------+---------------------+
| | | | | Struct1 (Structure) |
| 0 | size | 2 | 02 | uint8 |
| | array | | | List[int] |
| 1 | [0] | 1 | 01 | uint8 |
| 2 | [1] | 2 | 02 | uint8 |
| 3 | bookend | 153 | 99 | uint8 |
+--------+---------+-------+-------+---------------------+
>>>
>>> Struct1(array=[1, 2], bookend=0x99).ipack()
b'\x02\x01\x02\x99'
The association causes the size member to be updated when setting
a new array value using the array attribute:
>>> struct1 = Struct1(array=[1, 2], bookend=123)
>>> struct1.size
2
>>>
>>> struct1.array = [1, 2, 3, 4, 5]
>>> struct1.size
5
Using Ratios¶
The sized_member() function supports accepting a ratio parameter to facilitate
converting the size member units to a byte count when it is not a 1:1 ratio.
In the example below, a ratio of 2 means that for every size increment of
1, the array holds 2 extra bytes:
>>> class Struct2(Structure):
... size: int = member(fmt=uint8, compute=True)
... array: list = sized_member(size=size, fmt=array, ratio=2)
... bookend: int = member(fmt=uint8)
...
>>> Struct2.unpack(b'\x01\x01\x02\x99').dump()
+--------+---------+-------+-------+---------------------+
| Offset | Access | Value | Bytes | Format |
+--------+---------+-------+-------+---------------------+
| | | | | Struct2 (Structure) |
| 0 | size | 1 | 01 | uint8 |
| | array | | | List[int] |
| 1 | [0] | 1 | 01 | uint8 |
| 2 | [1] | 2 | 02 | uint8 |
| 3 | bookend | 153 | 99 | uint8 |
+--------+---------+-------+-------+---------------------+
>>>
>>> Struct2(array=[1, 2], bookend=0x99).ipack()
b'\x01\x01\x02\x99'
Ignoring Members¶
member() and sized_member() member definition functions
support marking the members to be ignored during comparisons and may
be used at the same time for that purpose:
>>> class Struct4(Structure):
... size: int = member(fmt=uint8, compute=True, ignore=True)
... array: list = sized_member(size=size, fmt=array, ignore=True)
... bookend: int = member(fmt=uint8)
...
>>> struct4 = Struct4.unpack(b'\x02\x01\x02\x99')
>>> struct4.dump()
+--------+---------+-------+-------+---------------------+
| Offset | Access | Value | Bytes | Format |
+--------+---------+-------+-------+---------------------+
| | | | | Struct4 (Structure) |
| 0 | size | 2 | 02 | uint8 |
| | array | | | List[int] |
| 1 | [0] | 1 | 01 | uint8 |
| 2 | [1] | 2 | 02 | uint8 |
| 3 | bookend | 153 | 99 | uint8 |
+--------+---------+-------+-------+---------------------+
>>>
>>> struct4 == Struct4(array=[], bookend=0x99)
True
Defaulting Sized Member¶
Specifying a default value for the array in the member definition allows the structure to be instantiated or packed without a value provided for the array:
>>> class Struct5(Structure):
... size: int = member(fmt=uint8, compute=True)
... array: list = sized_member(size=size, fmt=array, default=(0, 1))
... bookend: int = member(fmt=uint8)
...
>>> Struct5(bookend=0x99).dump()
+--------+---------+-------+-------+---------------------+
| Offset | Access | Value | Bytes | Format |
+--------+---------+-------+-------+---------------------+
| | | | | Struct5 (Structure) |
| 0 | size | 2 | 02 | uint8 |
| | array | | | List[int] |
| 1 | [0] | 0 | 00 | uint8 |
| 2 | [1] | 1 | 01 | uint8 |
| 3 | bookend | 153 | 99 | uint8 |
+--------+---------+-------+-------+---------------------+
>>>
>>> Struct5(bookend=0x99).ipack()
b'\x02\x00\x01\x99'
Accommodating Fixed Offset¶
The offset argument of the sized_member() function compensates for
the size member reflecting both the size of the variable size member
and overhead bytes such as the size member itself. For example,
the size member in the following structure reflects the total size
of the structure including the size member:
>>> from plum.array import ArrayX
>>> from plum.structure import Structure, member, sized_member
>>>
>>> array = ArrayX(fmt=uint8) # greedy array
>>>
>>> class Struct6(Structure):
... size: int = member(fmt=uint8, compute=True)
... array: list = sized_member(fmt=array, size=size, offset=1)
...
>>> Struct6(array=[0, 1]).dump()
+--------+--------+-------+-------+---------------------+
| Offset | Access | Value | Bytes | Format |
+--------+--------+-------+-------+---------------------+
| | | | | Struct6 (Structure) |
| 0 | size | 3 | 03 | uint8 |
| | array | | | List[int] |
| 1 | [0] | 0 | 00 | uint8 |
| 2 | [1] | 1 | 01 | uint8 |
+--------+--------+-------+-------+---------------------+