[plum] Tutorial: Dynamically Typed Structure Member

This tutorial shows how to use features of the Member class within a structure definition to control one member’s type based on the value of another structure member.

>>> from plum.littleendian import uint8, uint16, uint32
>>> from plum.structure import Structure, member

Basics

To demonstrate, examine the example below. The datatype member controls the data member’s type. The FMT_MAP dictionary’s __getitem__() method serves as a format factory and the fmt_arg defines the member value that is passed to the format factory function.

During unpacking of the structure, the the value of the unpacked datatype member determines the format of the data member per the defined mapping:

>>> FMT_MAP = {0: uint8, 1: uint16, 2: uint32}
>>>
>>> class Struct1(Structure):
...     datatype: int = member(fmt=uint8)
...     data: int = member(fmt=FMT_MAP.__getitem__, fmt_arg=datatype)
...
>>> struct1 = Struct1.unpack(b'\x01\x02\x00')
>>> struct1
Struct1(datatype=1, data=2)
>>> struct1.dump()
+--------+----------+-------+-------+---------------------+
| Offset | Access   | Value | Bytes | Format              |
+--------+----------+-------+-------+---------------------+
|        |          |       |       | Struct1 (Structure) |
| 0      | datatype | 1     | 01    | uint8               |
| 1      | data     | 2     | 02 00 | uint16              |
+--------+----------+-------+-------+---------------------+

Note

When fmt_arg is specified, the format factory is passed just the value of the specified structure member. When left unspecified, the entire structure instances is passed giving access to all available members (as shown in the example in the next section).

Similarly in packing operations, the value of the datatype member determines the format of the data member bytes:

>>> # datatype -> uint8 (0)
>>> Struct1(datatype=0, data=2).ipack()
b'\x00\x02'
>>>
>>> # datatype -> uint16 (1)
>>> Struct1(datatype=1, data=2).ipack()
b'\x01\x02\x00'

Custom Variable Member Type

The Member definition class accepts factory functions (or properties) for both the fmt and default arguments. The following example shows a more sophisticated application of factory functions that determine the type based the magnitude of the value.

>>> class Sample(Structure):
...
...     def determine_datatype(self):
...         return 1 if self.data >= 256 else 0
...
...     @property
...     def data_cls(self):
...         return uint16 if self.datatype else uint8
...
...     datatype: int = member(fmt=uint8, default=determine_datatype)
...     data: int = member(fmt=data_cls)
...
...     @data.setter
...     def data(self, value):
...         self[1] = value  # use index to avoid recursion
...         self.datatype = self.determine_datatype()

The data_cls property (which could have been a simple function by leaving @property decorator off) used as the fmt of the data member controls the class used when packing and unpacking the data member. The data_cls factory function facilitates leaving the datatype member unspecified when instantiation or packing and having it determined based on the value magnitude in the data member. For example:

>>> # integer that fits in a uint8
>>> Sample(data=0).dump()
+--------+----------+-------+-------+--------------------+
| Offset | Access   | Value | Bytes | Format             |
+--------+----------+-------+-------+--------------------+
|        |          |       |       | Sample (Structure) |
| 0      | datatype | 0     | 00    | uint8              |
| 1      | data     | 0     | 00    | uint8              |
+--------+----------+-------+-------+--------------------+
>>>
>>> # integer that requires a uint16
>>> Sample(data=256).dump()
+--------+----------+-------+-------+--------------------+
| Offset | Access   | Value | Bytes | Format             |
+--------+----------+-------+-------+--------------------+
|        |          |       |       | Sample (Structure) |
| 0      | datatype | 1     | 01    | uint8              |
| 1      | data     | 256   | 00 01 | uint16             |
+--------+----------+-------+-------+--------------------+

The setter decorator of the data property overrides the normal attribute set mechanism for the data member. In addition to storing the new value, it also updates the datatype member to reflect the new data value.

>>> # 'datatype' starts out as uint8
>>> sample = Sample(data=0)
>>> sample.dump()
+--------+----------+-------+-------+--------------------+
| Offset | Access   | Value | Bytes | Format             |
+--------+----------+-------+-------+--------------------+
|        |          |       |       | Sample (Structure) |
| 0      | datatype | 0     | 00    | uint8              |
| 1      | data     | 0     | 00    | uint8              |
+--------+----------+-------+-------+--------------------+
>>>
>>> # but changes based on setting 'data' attribute to a new value
>>> sample.data = 256
>>> sample.dump()
+--------+----------+-------+-------+--------------------+
| Offset | Access   | Value | Bytes | Format             |
+--------+----------+-------+-------+--------------------+
|        |          |       |       | Sample (Structure) |
| 0      | datatype | 1     | 01    | uint8              |
| 1      | data     | 256   | 00 01 | uint16             |
+--------+----------+-------+-------+--------------------+

Instead of providing a default factory function for the datatype member, consider providing a complete implementation for __init__ (rather than relying on the __init__ that would otherwise be generated). Besides being more explicit, it helps IDEs offer type ahead hints when instantiating the structure:

>>> class Sample(Structure):
...
...     def __init__(self, data, datatype=None):
...         if datatype is None:
...             datatype = 1 if data >> 8 else 0
...
...         self[:] = datatype, data
...
...     @property
...     def data_cls(self):
...         return uint16 if self.datatype else uint8
...
...     datatype: int = member(fmt=uint8)  # no default
...     data: int = member(fmt=data_cls)
...
...     @data.setter
...     def data(self, value):
...         self[1] = value  # use index to avoid recursion
...         self.datatype = 1 if value >> 8 else 0
...
>>> # and works the same
>>> sample = Sample(data=0)
>>> sample.dump()
+--------+----------+-------+-------+--------------------+
| Offset | Access   | Value | Bytes | Format             |
+--------+----------+-------+-------+--------------------+
|        |          |       |       | Sample (Structure) |
| 0      | datatype | 0     | 00    | uint8              |
| 1      | data     | 0     | 00    | uint8              |
+--------+----------+-------+-------+--------------------+
>>>
>>> sample.data = 256
>>> sample.dump()
+--------+----------+-------+-------+--------------------+
| Offset | Access   | Value | Bytes | Format             |
+--------+----------+-------+-------+--------------------+
|        |          |       |       | Sample (Structure) |
| 0      | datatype | 1     | 01    | uint8              |
| 1      | data     | 256   | 00 01 | uint16             |
+--------+----------+-------+-------+--------------------+