[plum.int.bitfields] Tutorial: Using Integer Bit Fields

Integer bit field (BitFields) types resemble Structure types. Both provide mechanisms to pack or unpack sequences of bytes and conveniently specify or access data within the bytes. But BitFields types provide access to individual bits and/or bit fields within the bytes buffer. Bit fields may span byte boundaries. BitFields types provide integer characteristics as well. This tutorial teaches how to create and use integer bit fields plum types.

Creating an Integer Bit Fields Type

To create a basic integer bit fields type, subclass the BitFields class specifying the total number of bytes and byte order. Use class attribute type annotation syntax to define the bit fields. Specify the bit field name, followed by a colon and a type (any int-like type), then an equal sign and finally the position and number of bits of the bit field. For example:

>>> from plum.int.bitfields import BitFields, BitField
>>>
>>> class Sample(BitFields, nbytes=1, byteorder='little'):
...     boolean_flag: bool = BitField(pos=0, size=1)
...     integer_flag: int = BitField(pos=1, size=1)
...     nibble: int = BitField(pos=2, size=4)
...     reserved: int = BitField(pos=6, size=2)
...
>>>

Note

The bit field definition accepts a cls argument to specify the type. This facilitates skipping the type annotation or providing a different type annotation. For example:

>>> class Sample(BitFields, nbytes=1, byteorder='little'):
...     boolean_flag = BitField(cls=bool, pos=0, size=1)
...     integer_flag = BitField(cls=int, pos=1, size=1)
...     nibble = BitField(cls=int, pos=2, size=4)
...     reserved = BitField(cls=int, pos=6, size=2)
...
>>>

Unpacking Bytes

plum integer bit fields types (classes) convert bytes into an instance if the class when used with the various plum unpacking mechanisms. For example (using the enumeration from the previous section):

>>> from plum import unpack, Buffer
>>>
>>> # utility function
>>> unpack(Sample, b'\x00')
Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>>
>>> # class method
>>> Sample.unpack(b'\x00')
Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>>
>>> # bytes buffer
>>> with Buffer(b'\x00') as buffer:
...     sample = buffer.unpack(Sample)
...
>>> sample
Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)

The dump property offers a summary of the bit fields providing a hint of the access method:

>>> sample.dump()
+--------+---------------+-------+-------+--------+
| Offset | Access        | Value | Bytes | Type   |
+--------+---------------+-------+-------+--------+
| 0      |               | 0     | 00    | Sample |
|  [0]   | .boolean_flag | False |       | bool   |
|  [1]   | .integer_flag | 0     |       | int    |
|  [2:5] | .nibble       | 0     |       | int    |
|  [6:7] | .reserved     | 0     |       | int    |
+--------+---------------+-------+-------+--------+

As the entries in the access column suggest, the bit fields instance offers access to the individual bit fields through the named attributes:

>>> sample.boolean_flag
False
>>> sample.nibble
0

Instantiation

A bit fields integer type constructor bears similarity to a dict constructor:

>>> # pass values as keyword arguments (argument names match bit field names)
>>> Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>>
>>> # pass dictionary of values as positional argument (keys match bit field names)
>>> Sample({'boolean_flag': False, 'integer_flag': 0, 'nibble': 0, 'reserved': 0})
Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>>
>>> # pass iterable of key/value pairs as positional argument (keys match bit field names)
>>> Sample([('boolean_flag', False), ('integer_flag', 0), ('nibble', 0), ('reserved', 0)])
Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>>
>>> # pass both positional argument and keyword arguments (keyword arguments trump positional)
>>> Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)

Additionally, the constructor accepts an integer value as a positional argument to set the value of all bit fields in one fell swoop:

>>> Sample(0)
Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)

Instance Properties

Regardless of whether the integer bit fields type was instantiated or created with one of the unpack mechanisms, its properties are the same. The instance has integer characteristics:

>>> sample = Sample(1)
>>>
>>> sample == 1  # comparisons
True
>>>
>>> sample + 1  # arithmetic operations
2
>>> sample << 1  # logical operations
2
>>> int(sample)  # conversions
1

Bit fields are settable by attribute assignment:

>>> sample = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>> int(sample)
0
>>> sample.boolean_flag = True
>>> sample
Sample(boolean_flag=True, integer_flag=0, nibble=0, reserved=0)
>>> int(sample)
1

They support comparison against other instances or against dictionaries:

>>> # instance vs. instance
>>> sample1 = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>> sample2 = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>> sample1 == sample2
True
>>> # instance vs. dictionary
>>> sample1 = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>> sample2 = {'boolean_flag': False, 'integer_flag': 0, 'nibble': 0, 'reserved': 0}
>>> sample1 == sample2
True

The asdict() method supports obtaining bit field values in dictionary form:

>>> sample = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>> sample.asdict()
{'boolean_flag': False, 'integer_flag': 0, 'nibble': 0, 'reserved': 0}

The update() method supports altering a number of bitfields similar to the dict update() method:

>>> sample = Sample(boolean_flag=False, integer_flag=0, nibble=7, reserved=3)
>>> sample.update(nibble=0, reserved=0)
>>> sample
Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)

Packing Bytes

plum integer bit fields types convert integers (or anything int-like, or anything that the constructor accepts) into bytes when used with the various plum packing mechanisms. For example:

>>> from plum import pack
>>>
>>> # utility function (with an integer, but could be a dictionary)
>>> pack(Sample, 0)
bytearray(b'\x00')
>>>
>>> # class method (with a dict, but could be an integer)
>>> Sample.pack({'boolean_flag': False, 'integer_flag': 0, 'nibble': 0, 'reserved': 0})
bytearray(b'\x00')
>>>
>>> # instance method (with just one of the many ways to instantiate)
>>> Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0).pack()
bytearray(b'\x00')

Defaulting a Bit Field

To specify a default value to be utilized if a bit field value is not provided during class instantiation (or packing), use the default argument of the BitField definition. For example:

>>> from plum.int.bitfields import BitFields, BitField
>>>
>>> class Sample(BitFields, nbytes=1, byteorder='little'):
...     boolean_flag: bool = BitField(pos=0, size=1)
...     integer_flag: int = BitField(pos=1, size=1)
...     nibble: int = BitField(pos=2, size=4)
...     reserved: int = BitField(pos=6, size=2, default=3)
...
>>> # 'reserved' bit field not specified
>>> Sample(boolean_flag=False, integer_flag=0, nibble=0)
Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=3)

To specify a default value for undefined bit fields, use the default argument in the class definition. When instantiating (or packing), this provides a starting value before supplied (or defaulted) bit field values are applied. Normally the starting value defaults to zero when left unspecified. The following example shows the result of a default value of 0xc0 (bits 6 and 7 set):

>>> class Sample(BitFields, nbytes=1, byteorder='little', default=0xc0):
...     boolean_flag: bool = BitField(pos=0, size=1)
...     integer_flag: int = BitField(pos=1, size=1)
...     nibble: int = BitField(pos=2, size=4)
...
>>> Sample(boolean_flag=False, integer_flag=0, nibble=0).pack()
bytearray(b'\xc0')

Ignoring a Bit Field

To specify a bit field to be ignored during comparisons, set the ignore argument to True in the BitField definition. For example:

>>> from plum.int.bitfields import BitFields, BitField
>>>
>>> class Sample(BitFields, nbytes=1, byteorder='little'):
...     boolean_flag: bool = BitField(pos=0, size=1)
...     integer_flag: int = BitField(pos=1, size=1)
...     nibble: int = BitField(pos=2, size=4)
...     reserved: int = BitField(pos=6, size=2, ignore=True)
...
>>> sample1 = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>> sample2 = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=3)
>>> sample1 == sample2
True
>>> # use asdict() method to compare for exactly
>>> sample1.asdict() == sample2.asdict()
False

To specify undefined bit fields to be ignored during comparisons, provide a bit mask (with bits set to be ignored) with the ignore argument in the class definition. The following example shows the result of a ignoring bits 6 and 7 by supplying the bit mask value of 0xc0:

>>> class Sample(BitFields, nbytes=1, byteorder='little', ignore=0xc0):
...     boolean_flag: bool = BitField(pos=0, size=1)
...     integer_flag: int = BitField(pos=1, size=1)
...     nibble: int = BitField(pos=2, size=4)
...
>>> sample1 = Sample(0x00)
>>> sample2 = Sample(0x80)
>>> sample1 == sample2
True
>>> # use int( ) conversion to compare exactly
>>> int(sample1) == int(sample2)
False

Alternative Bit Field Types and Nesting

Bit field definitions support any Python type that supports integer conversion. The most popular choices besides int, are booleans and enumerations as demonstrated in the following example:

>>> from enum import IntFlag
>>> from plum.int.bitfields import BitFields, BitField
>>>
>>> class Letter(IntFlag):  # use IntFlag to tolerate undefined values
...
...     """Sample IntEnum."""
...
...     A = 1
...     B = 2
>>>
>>> class Sample(BitFields, nbytes=1):
...
...     """Sample BitFields subclass."""
...
...     letter: Letter = BitField(pos=0, size=4)
...     boolean: bool = BitField(pos=4, size=1)
...
>>> sample = Sample()
>>> sample.boolean
False
>>> sample.letter
<Letter.0: 0>
>>>
>>> sample.update(boolean=True, letter=Letter.A)
>>> sample.dump()
+--------+----------+----------+-------+--------+
| Offset | Access   | Value    | Bytes | Type   |
+--------+----------+----------+-------+--------+
| 0      |          | 17       | 11    | Sample |
|  [0:3] | .letter  | Letter.A |       | Letter |
|  [4]   | .boolean | True     |       | bool   |
+--------+----------+----------+-------+--------+

Bit field definitions also support nested plum bit fields types as demonstrated next:

>>> class Inner(BitFields, nbytes=1):
...     a: int = BitField(pos=0, size=2)
...     b: int = BitField(pos=2, size=2)
...
>>>
>>> class Outer(BitFields, nbytes=1):
...     h: int = BitField(pos=0, size=2)
...     i: Inner = BitField(pos=2, size=4)
...     j: int = BitField(pos=6, size=2)
...
>>> sample = Outer(h=0, i={'a': 1, 'b': 2}, j=3)
>>> sample.i
Inner(a=1, b=2)
>>> sample.i.a
1
>>> sample.dump()
+--------+--------+-------+-------+-------+
| Offset | Access | Value | Bytes | Type  |
+--------+--------+-------+-------+-------+
| 0      |        | 228   | e4    | Outer |
|  [0:1] | .h     | 0     |       | int   |
|        | .i     |       |       | Inner |
|  [2:3] |   .a   | 1     |       | int   |
|  [4:5] |   .b   | 2     |       | int   |
|  [6:7] | .j     | 3     |       | int   |
+--------+--------+-------+-------+-------+
>>>
>>> sample == {'h': 0, 'i': 9, 'j': 3}
True

Automatic Bit Field Positions

Bit fields automatically position themselves when pos is not specified. The first bit field assumes a bit position of zero. Subsequent bit fields assume a position immediately following the previous bit field. For example:

>>> class AutoMix(BitFields, nbytes=1):
...
...     f1: int = BitField(size=2)
...     f2: int = BitField(size=4)
...     f3: int = BitField(size=1, pos=7)  # explicit position
...
>>> AutoMix(0x85).dump()
+--------+--------+-------+-------+---------+
| Offset | Access | Value | Bytes | Type    |
+--------+--------+-------+-------+---------+
| 0      |        | 133   | 85    | AutoMix |
|  [0:1] | .f1    | 1     |       | int     |
|  [2:5] | .f2    | 1     |       | int     |
|  [7]   | .f3    | 1     |       | int     |
+--------+--------+-------+-------+---------+