[plum] Tutorial: Bit Field Structure Members

The BitFields data store type provides a mechanism to embed, or nest a collection of bit fields within a single structure member. This tutorial demonstrates an alternative that avoids nesting, using the bitfield_member() function to define structure members as bit fields (members that hold sequences of bits).

The examples shown on this page require the following setup:

>>> from enum import IntEnum
>>> from plum.bigendian import uint8
>>> from plum.enum import EnumX
>>> from plum.structure import Structure, bitfield_member, member
>>> from plum.utilities import pack, unpack
>>>
>>> class Register(IntEnum):
...     PC = 0
...     SP = 1
...     R0 = 2
...

Basic Usage

The member() function accepts a fmt that provides the transform between a structure member value and bytes. The func:bitfield_member function instead accepts size, lsb, and typ arguments to define a transform between a structure member value and a sequence of bits (typ argument defaults to a Python int):

>>> class Sample(Structure):
...    nibble1: int = bitfield_member(size=4, lsb=4)
...    nibble2: int = bitfield_member(size=4, lsb=0)
...    byte: int = member(fmt=uint8)
...
>>> pack(Sample(nibble1=1, nibble2=2, byte=3)).hex()
'1203'

In applications such as the example above where there is no gap between the bit fields, let the lsb parameter default to None. Then the bit field member automatically calculates the position based on the position and size of the bit fields that follow. You may also let the typ parameter default. If a type annotation exists, the bit field member uses it, otherwise it defaults to int:

>>> class Sample(Structure):
...    nibble1: int = bitfield_member(size=4)
...    nibble2: int = bitfield_member(size=4)
...    byte: int = member(fmt=uint8)
...
>>> pack(Sample(nibble1=1, nibble2=2, byte=3)).hex()
'1203'

In the example above, the bitfield order defaulted to be from most significant to the least. To reverse, set fieldorder="least_to_most":

>>> class Sample(Structure, fieldorder="least_to_most"):
...    nibble1: int = bitfield_member(size=4)
...    nibble2: int = bitfield_member(size=4)
...    byte: int = member(fmt=uint8)
...
>>> pack(Sample(nibble1=1, nibble2=2, byte=3)).hex()
'2103'

For typ, you may use any callable that accepts an integer and produces something capable of being converted to an integer (ie. int() can be called with it as an argument). Examples include a simple integer enumeration type, an integer flag enumeration type, or even a custom subclass of BitFields. The following examples shows use of a simple enumeration type:

>>> class Sample(Structure):
...    reg1: Register = bitfield_member(lsb=0, size=4, typ=Register)
...    reg2: int = bitfield_member(lsb=4, size=4, typ=EnumX(Register, strict=False))
...    byte: int = member(fmt=uint8)
...
>>> sample = unpack(Sample, b'\x21\x03')
>>> sample.dump()
+--------+--------+-------------+----------+--------------------+
| Offset | Access | Value       | Bytes    | Format             |
+--------+--------+-------------+----------+--------------------+
|        |        |             |          | Sample (Structure) |
| 0      |        | 33          | 21       |                    |
|  [0:4] | reg1   | Register.SP | ....0001 | Register           |
|  [4:8] | reg2   | Register.R0 | 0010.... | Register (IntEnum) |
| 1      | byte   | 3           | 03       | uint8              |
+--------+--------+-------------+----------+--------------------+
>>>
>>> # non-strict enum transform allows integers not in enumeration to come through
>>> sample = unpack(Sample, b'\x31\x00')
>>> sample.reg2
3

For bit fields that store a signed integer, set signed=True:

>>> class Sample(Structure):
...    nibble1: int = bitfield_member(lsb=0, size=4, signed=True)
...    nibble2: int = bitfield_member(lsb=4, size=4)
...    byte: int = member(fmt=uint8)
...
>>> pack(Sample(nibble1=-1, nibble2=0, byte=3)).hex()
'0f03'

The bitfield_member() function accepts the following keyword arguments that have the same behaviors as those in a normal member() function:

default:default value to use in initializer when one not passed ignore:ignore member when comparing against another structure instance readonly:block setting member attribute compute:automatically compute based on another member value (use in combination with sized_member(), dimmed_member(), etc.)

Reserving Bits

The bitfield_member() function nbytes parameter controls the number of bytes the bit field (and all bit fields that immediately follow that don’t set the nbytes explicitly) occupy. When left unspecified, the number of bytes becomes just large enough to accommodate the bit fields. Within the first bit field member defiinition, specify nbytes explicitly to reserve extra bits without defining an additional member for them:

>>> class Sample(Structure):
...    nibble1: int = bitfield_member(lsb=0, size=4, nbytes=2)
...    nibble2: int = bitfield_member(lsb=4, size=4)
...    byte: int = member(fmt=uint8)
...
>>> pack(Sample(nibble1=1, nibble2=2, byte=3)).hex()
'210003'

When leaving lsb unspecified, setting nbytes a second time has the net effect of resetting the automatically computed position to honor the first nbytes. The following example demonstrates two equivalent structures, one explicitly specifying lsb, one not.

>>> class Sample1(Structure):
...    nibble1: int = bitfield_member(size=4, nbytes=2)
...    nibble2: int = bitfield_member(size=4, nbytes=2)
...    byte: int = member(fmt=uint8)
...
>>> class Sample2(Structure):
...    nibble1: int = bitfield_member(lsb=0, size=4, nbytes=4)
...    nibble2: int = bitfield_member(lsb=16, size=4)
...    byte: int = member(fmt=uint8)
...
>>> sample1 = Sample1(nibble1=1, nibble2=2, byte=3)
>>> sample2 = Sample2(nibble1=1, nibble2=2, byte=3)
>>> pack(sample1) == pack(sample2)
True

Tip

The Sample2 has slightly better performance.

Byte Order

Structure members defined with the member() function follow the byte order of the format specified with the fmt argument. Since the bitfield_member() function does not accept a format, the structure controls the byte order of the bit fields. By default, structures use little endian. The Structure metaclass accepts a byteorder keyword argument to change the byteorder to big endian.

>>> class BigSample(Structure, byteorder="big"):
...    nibble1: int = bitfield_member(lsb=0, size=4, nbytes=2)
...    nibble2: int = bitfield_member(lsb=4, size=4)
...    byte: int = member(fmt=uint8)
...
>>> pack(BigSample(nibble1=1, nibble2=2, byte=3)).hex()
'002103'
>>> class LittleSample(Structure):
...    nibble1: int = bitfield_member(lsb=0, size=4, nbytes=2)
...    nibble2: int = bitfield_member(lsb=4, size=4)
...    byte: int = member(fmt=uint8)
...
>>> pack(LittleSample(nibble1=1, nibble2=2, byte=3)).hex()
'210003'