Bit fields data store type.

[plum.bitfields] Module Reference

The plum.bitfields module provides the BitFields class which acts as both a transform and a data store. As a data store of a collection of bit fields (sequences of bits), instances provide access to the individual bit field or as an integer which comprises all the bit fields. As a transform, the class converts instances into bytes and bytes into instances. This reference page demonstrates creating and using a BitFields data store as well as provides API details.

The examples shown on this page require the following setup:

>>> from plum.bigendian import uint8
>>> from plum.bitfields import BitFields, bitfield
>>> from plum.structure import Structure, member
>>> from plum.utilities import pack, unpack

Basic Usage

To create your own bit fields definition, subclass BitFields. Within the class body, use the bitfield() function for each bit field specifying the Python type (if something other than a Python int), bit position, and size (in number of bits).

>>> class Sample(BitFields):
...     reserved: int = bitfield(lsb=6, size=2)
...     nibble: int = bitfield(lsb=2, size=4)
...     integer_flag: int = bitfield(lsb=1, size=1)
...     boolean_flag: bool = bitfield(typ=bool, lsb=0, size=1)
...

When the Sample subclass is used as the format fmt argument, unpacking converts bytes into an instance of this class.

>>> fmt = Sample
>>> sample = unpack(fmt, b'\x3d')
>>> sample
Sample(reserved=0, nibble=15, integer_flag=0, boolean_flag=True)

Since the BitFields subclass instances directly support the transform protocol, no format needs to be specified when packing:

>>> pack(sample).hex()
'3d'

Integers holding the cumulative value of all bit fields may be packed directly when the BitFields subclass is used as the format (no subclass instantiation required):

>>> pack(0x3d, fmt=Sample).hex()
'3d'

The bitfield() function defined in the Sample subclass body provides access to the values of the individual bitfields as attributes:

>>> sample.boolean_flag
True
>>> sample.nibble
15

A meta class (the class which constructs BitFields subclasses) creates a custom instance initializer for each subclass. The initializer accepts a value for each bit field as keyword arguments:

>>> sample = Sample(boolean_flag=True, integer_flag=0, nibble=15, reserved=0)
>>> sample
Sample(reserved=0, nibble=15, integer_flag=0, boolean_flag=True)

The from_int() factory also accepts an integer value that reflects the value of all bit fields:

>>> sample = Sample.from_int(0x3d)
>>> sample
Sample(reserved=0, nibble=15, integer_flag=0, boolean_flag=True)

The dump produced by using pack_and_dump(), unpack_and_dump(), and the dump() method includes breakdowns for each bit field and lists the attribute name as the “Access”.

>>> sample.dump()
+--------+--------------+-------+----------+--------------------+
| Offset | Access       | Value | Bytes    | Format             |
+--------+--------------+-------+----------+--------------------+
| 0      |              | 61    | 3d       | Sample (BitFields) |
|  [6:8] | reserved     | 0     | 00...... | int                |
|  [2:6] | nibble       | 15    | ..1111.. | int                |
|  [1]   | integer_flag | 0     | ......0. | int                |
|  [0]   | boolean_flag | True  | .......1 | bool               |
+--------+--------------+-------+----------+--------------------+

BitFields subclassses also support getting and setting individual bits by integer indexing:

>>> sample = Sample.from_int(0)
>>>
>>> sample[0]
False
>>> sample[0] = True
>>> int(sample)
1
>>>
>>> sample[2:8]
[False, False, False, False, False, False]
>>> sample[2:8] = [False, False, False, False, False, True]
>>> int(sample)
129

Instance Properties

Regardless of whether the integer bit fields type was instantiated or created with one of the unpack mechanisms, its properties are the same. The instance has integer characteristics:

>>> sample = Sample.from_int(1)
>>>
>>> sample == 1  # comparisons
True
>>>
>>> sample + 1  # arithmetic operations
2
>>> sample << 1  # logical operations
2
>>> int(sample)  # conversions
1

Bit fields are settable by attribute assignment:

>>> sample = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>> int(sample)
0
>>> sample.boolean_flag = True
>>> sample
Sample(reserved=0, nibble=0, integer_flag=0, boolean_flag=True)
>>> int(sample)
1

They support comparison against other instances:

>>> # instance vs. instance
>>> sample1 = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>> sample2 = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>> sample1 == sample2
True

The asdict() method supports obtaining bit field values in dictionary form:

>>> sample = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>> sample.asdict()
{'reserved': 0, 'nibble': 0, 'integer_flag': 0, 'boolean_flag': False}

Defaulting a Bit Field

To specify a default value to be utilized if a bit field value is not provided during class instantiation (or packing), use the default argument of the bitfield() definition. For example:

>>> class Sample(BitFields):
...     reserved: int = bitfield(lsb=6, size=2, default=3)
...     nibble: int = bitfield(lsb=2, size=4)
...     integer_flag: int = bitfield(lsb=1, size=1)
...     boolean_flag: bool = bitfield(typ=bool, lsb=0, size=1)
...
>>> # 'reserved' bit field not specified
>>> Sample(boolean_flag=False, integer_flag=0, nibble=0)
Sample(reserved=3, nibble=0, integer_flag=0, boolean_flag=False)

To specify a default value for undefined bit fields, use the default argument in the class definition. When instantiating (or packing), this provides a starting value before supplied (or defaulted) bit field values are applied. Normally the starting value defaults to zero when left unspecified. The following example shows the result of a default value of 0xc0 (bits 6 and 7 set):

>>> class Sample(BitFields, default=0xc0):
...     nibble: int = bitfield(lsb=2, size=4)
...     integer_flag: int = bitfield(lsb=1, size=1)
...     boolean_flag: bool = bitfield(lsb=0, size=1)
...
>>> Sample(boolean_flag=False, integer_flag=0, nibble=0).ipack()
b'\xc0'

Ignoring a Bit Field

To specify a bit field to be ignored during comparisons, set the ignore argument to True in the bitfield() definition. For example:

>>> from plum.bitfields import BitFields, bitfield
>>>
>>> class Sample(BitFields):
...     reserved: int = bitfield(lsb=6, size=2, ignore=True)
...     nibble: int = bitfield(lsb=2, size=4)
...     integer_flag: int = bitfield(lsb=1, size=1)
...     boolean_flag: bool = bitfield(lsb=0, size=1)
...
>>> sample1 = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=0)
>>> sample2 = Sample(boolean_flag=False, integer_flag=0, nibble=0, reserved=3)
>>> sample1 == sample2
True
>>> # use asdict() method to compare for exact equality
>>> sample1.asdict() == sample2.asdict()
False

To specify undefined bit fields to be ignored during comparisons, provide a bit mask (with bits set to be ignored) with the ignore argument in the class definition. The following example shows the result of a ignoring bits 6 and 7 by supplying the bit mask value of 0xc0:

>>> class Sample(BitFields, ignore=0xc0):
...     nibble: int = bitfield(lsb=2, size=4)
...     integer_flag: int = bitfield(lsb=1, size=1)
...     boolean_flag: bool = bitfield(lsb=0, size=1)
...
>>> sample1 = Sample.from_int(0x00)
>>> sample2 = Sample.from_int(0x80)
>>> sample1 == sample2
True
>>> # use int( ) conversion to compare exactly
>>> int(sample1) == int(sample2)
False

Alternative Bit Field Types and Nesting

Bit field definitions support any Python type that supports integer conversion. The most popular choices besides int, are booleans and enumerations as demonstrated in the following example:

>>> from enum import IntEnum
>>> from plum.bitfields import bitfield, BitFields
>>> from plum.enum import EnumX
>>>
>>> class Letter(IntEnum):
...
...     """Sample IntEnum."""
...
...     A = 0
...     B = 1
>>>
>>> flexible_letter = EnumX(Letter, strict=False, nbytes=1)
>>>
>>> class Sample(BitFields):
...
...     """Sample BitFields subclass."""
...
...     tolerant: int = bitfield(typ=flexible_letter, lsb=4, size=2)
...     strict: Letter = bitfield(typ=Letter, lsb=2, size=2)
...     boolean: bool = bitfield(typ=bool, lsb=0, size=1)
...
>>> sample = unpack(Sample, b'\x00')
>>> sample.boolean
False
>>> sample.strict
<Letter.A: 0>
>>> sample.tolerant
<Letter.A: 0>
>>>
>>> # non-strict enum transform allows integers not in enumeration to come through
>>> sample = unpack(Sample, b'\x30')
>>> sample.tolerant
3

Bit field definitions also support nested BitFields subclasses as long as the nested bitfield is defined using nested=True:

>>> class Inner(BitFields, nested=True):
...     b: int = bitfield(lsb=2, size=2)
...     a: int = bitfield(lsb=0, size=2)
...
>>>
>>> class Outer(BitFields):
...     j: int = bitfield(lsb=6, size=2)
...     i: Inner = bitfield(typ=Inner, lsb=2, size=4)
...     h: int = bitfield(lsb=0, size=2)
...
>>> sample = Outer(j=3, i=Inner(a=1, b=2), h=0)
>>> sample.i
Inner(b=2, a=1)
>>> sample.i.a
1
>>> sample.dump()
+--------+--------+-------+----------+-------------------+
| Offset | Access | Value | Bytes    | Format            |
+--------+--------+-------+----------+-------------------+
| 0      |        | 228   | e4       | Outer (BitFields) |
|  [6:8] | j      | 3     | 11...... | int               |
|        | i      |       |          | Inner (BitFields) |
|  [4:6] |   b    | 2     | ..10.... | int               |
|  [2:4] |   a    | 1     | ....01.. | int               |
|  [0:2] | h      | 0     | ......00 | int               |
+--------+--------+-------+----------+-------------------+

Note

Setting nested=True includes extra logic when getting and setting bitfields in a nested bitfields object. The extra logic has no effect when using the bitfields type outside of a nested application other than a small performance degradation.

Automatic Bit Field Positions

Bit fields automatically position themselves when lsb is not specified. By default, the lsb gets computed by placing the bitfields in order from the most significant bit locations to the least. For example:

>>> class AutoMix(BitFields):
...
...     f1: int = bitfield(size=1)
...     f2: int = bitfield(size=5)
...     f3: int = bitfield(size=2)
...
>>> AutoMix.from_int(0x8b).dump()
+--------+--------+-------+----------+---------------------+
| Offset | Access | Value | Bytes    | Format              |
+--------+--------+-------+----------+---------------------+
| 0      |        | 139   | 8b       | AutoMix (BitFields) |
|  [7]   | f1     | 1     | 1....... | int                 |
|  [2:7] | f2     | 2     | .00010.. | int                 |
|  [0:2] | f3     | 3     | ......11 | int                 |
+--------+--------+-------+----------+---------------------+

set fieldorder="least_to_most" to automatically position the bit fields in order from the least significant bit locations to the most. For example:

>>> class AutoMix(BitFields, fieldorder="least_to_most"):
...
...     f1: int = bitfield(size=2)
...     f2: int = bitfield(size=5)
...     f3: int = bitfield(size=1)
...
>>> AutoMix.from_int(0x8b).dump()
+--------+--------+-------+----------+---------------------+
| Offset | Access | Value | Bytes    | Format              |
+--------+--------+-------+----------+---------------------+
| 0      |        | 139   | 8b       | AutoMix (BitFields) |
|  [0:2] | f1     | 3     | ......11 | int                 |
|  [2:7] | f2     | 2     | .00010.. | int                 |
|  [7]   | f3     | 1     | 1....... | int                 |
+--------+--------+-------+----------+---------------------+

Controlling Overall Size and Byte Order

By default, the number of bytes that the overall collection of bitfields occupy gets computed automatically based on the location and size of the bit field in the most significant position. Use the nbytes keyword argument when constructing the subclass to explicitly control the number of bytes.

The transform between the byte sequence and the integer used as the basis of the bitfields follows the little endian format. To use big endian, set byteorder="big". For example:

>>> class Sample(BitFields, nbytes=2, byteorder="big"):
...
...     f1: int = bitfield(size=4)
...     f2: int = bitfield(size=4)
...
>>> Sample.from_int(0x0012).dump()
+--------+--------+-------+-------------------+--------------------+
| Offset | Access | Value | Bytes             | Format             |
+--------+--------+-------+-------------------+--------------------+
| 0      |        | 18    | 00 12             | Sample (BitFields) |
|  [4:8] | f1     | 1     | ........ 0001.... | int                |
|  [0:4] | f2     | 2     | ........ ....0010 | int                |
+--------+--------+-------+-------------------+--------------------+

Example Structure Usage

>>> class TwoNibbles(BitFields):
...     nibble1: int = bitfield(size=4)
...     nibble2: int = bitfield(size=4)
...
>>> class Sample(Structure):
...    nibbles: TwoNibbles = member(fmt=TwoNibbles)
...    byte: int = member(fmt=uint8)
...
>>> sample = Sample(nibbles=TwoNibbles(nibble1=1, nibble2=2), byte=3)
>>> pack(sample).hex()
'1203'
>>>
>>> sample = unpack(Sample, b'\x21\x03')
>>> sample.dump()
+--------+-----------+-------+----------+------------------------+
| Offset | Access    | Value | Bytes    | Format                 |
+--------+-----------+-------+----------+------------------------+
|        |           |       |          | Sample (Structure)     |
| 0      | nibbles   | 33    | 21       | TwoNibbles (BitFields) |
|  [4:8] |   nibble1 | 2     | 0010.... | int                    |
|  [0:4] |   nibble2 | 1     | ....0001 | int                    |
| 1      | byte      | 3     | 03       | uint8                  |
+--------+-----------+-------+----------+------------------------+

Note

See Bit Field Structure Member tutorial for an alternative method of placing bit fields within a structure which avoids nesting.

API Reference

class plum.bitfields.BitFields

Bit fields data store type.

byteorder = 'little'

dump

Summary containing details of bytes and layout.

nbytes = 4

asdict()

Return bit field values in dictionary form.

Returns:bit field names/values Return type:dict

classmethod pack(value: Any) → bytes

Pack value as formatted bytes.

Raises:PackError if type error, value error, etc.

classmethod pack_and_dump(value: Any) → Tuple[bytes, plum.dump.Dump]

Pack value as formatted bytes and produce bytes summary.

Raises:PackError if type error, value error, etc.

classmethod unpack(buffer: bytes) → Any

Unpack value from formatted bytes.

Raises:UnpackError if insufficient bytes, excess bytes, or value error

classmethod unpack_and_dump(buffer: bytes) → Tuple[Any, plum.dump.Dump]

Unpack value from bytes and produce packed bytes summary.

Raises:UnpackError if insufficient bytes, excess bytes, or value error

plum.bitfields.bitfield(doc: str = '', *, size: int, typ: type = <class 'int'>, lsb: Optional[int] = None, default: Optional[int] = None, signed: bool = False, ignore: bool = False, readonly: bool = False, argrepr: Optional[str] = None) → Any

Define bit field.

Parameters:

• doc – one line description
• size – size in bits
• typ – bit field type
• lsb – bit offset of least significant bit
• default – initial value when unspecified
• signed – interpret as signed integer
• ignore – do not include field in comparisons
• readonly – block setting member attribute
• argrepr – format to represent member argument in structure repr