[plum] Tutorial: Packing Bytes

This tutorial demonstrates the various methods and capabilities of packing bytes from the convenient forms supported by the various plum types.

The tutorial uses the following representative plum type, but the examples apply to all plum types or derivatives of them.

>>> from plum.structure import Member, Structure
>>> from plum.int.big import UInt8, UInt16
>>>
>>> class Sample(Structure):
...     m1: int = Member(cls=UInt16)
...     m2: int = Member(cls=UInt8)
...
>>>

Pack Methods

[Plum Type Instance Method]

Off the shelf plum types (e.g. UInt16) or derivatives of plum types (e.g. Sample from above) offer a pack() method that packs the instance and returns the resulting bytearray. For example:

>>> # pack an off the shelf plum type instance into a bytearray
>>> UInt16(1).pack()
bytearray(b'\x00\x01')
>>>
>>> # pack a derivative plum type instance into a bytearray
>>> Sample(m1=1, m2=2).pack()
bytearray(b'\x00\x01\x02')

[Plum Type Class Method]

The pack() method also functions as a class method. Instead of instantiating the plum type, call the pack() method with the value to be packed:

>>> # off the shelf plum type packed into a bytearray
>>> UInt16.pack(1)
bytearray(b'\x00\x01')
>>>
>>> # derivative plum type packed into a bytearray
>>> Sample.pack({'m1': 1, 'm2': 2})
bytearray(b'\x00\x01\x02')

Pack Utility Function

The pack() utility function mirrors the standard library struct.pack() function except instead of accepting a format string, it accepts plum types:

>>> from plum import pack
>>> from plum.int.big import UInt16, UInt8
>>>
>>> # format as single plum type accepts a value as positional argument
>>> fmt = UInt8
>>> pack(fmt, 1)
bytearray(b'\x01')
>>>
>>> # format as tuple/list of types accepts values as positional arguments
>>> fmt = (UInt8, UInt16)
>>> pack(fmt, 1, 2)
bytearray(b'\x01\x00\x02')
>>>
>>> # format as dictionary of types accepts values as keyword arguments
>>> fmt = {'a': UInt8, 'b': UInt16}
>>> pack(fmt, a=1, b=2)
bytearray(b'\x01\x00\x02')

The pack() function supports packing arbitrarily nested values within by specifying the structure using tuples/lists and dictionaries within the format. For exammple:

>>> fmt = {'a': (UInt8, UInt8), 'b': {'i': UInt8, 'ii': UInt8}}
>>>
>>> pack(fmt, a=(1, 2), b={'i': 3, 'ii': 4})
bytearray(b'\x01\x02\x03\x04')

Packing Directly Within a Bytes Buffer

The pack_into() plum type method as well as the pack_into() utility function have the same packing behaviors discussed in the previous sections except the functions take required buffer and offset arguments. The buffer may be any writeable bytes-like buffer. The offset argument specifies the number of bytes from the beginning of the buffer to start placing packed item bytes.

>>> from plum import pack_into
>>> from plum.int.big import UInt8
>>>
>>> OFFSET = 1
>>>
>>> buffer = bytearray(3)
>>> buffer
bytearray(b'\x00\x00\x00')
>>>
>>> # Plum Type Instance Method
>>> UInt8(0xaa).pack_into(buffer, OFFSET)
>>> buffer
bytearray(b'\x00\xaa\x00')
>>>
>>> # Plum Type class Method
>>> UInt8.pack_into(buffer, OFFSET, 0xbb)
>>> buffer
bytearray(b'\x00\xbb\x00')
>>>
>>> # formatted utility function
>>> fmt = (UInt8, UInt8)
>>> pack_into(fmt, buffer, OFFSET, 0xcc, 0xdd)
>>> buffer
bytearray(b'\x00\xcc\xdd')

Caveats

Specifying an offset outside the bounds of the byte buffer results in an exception:

>>> buffer = bytearray(2)
>>> buffer
bytearray(b'\x00\x00')
>>>
>>> pack_into(UInt8, buffer, 3, 0x99)
Traceback (most recent call last):
    ...
plum._exceptions.PackError: offset 3 out of range for 2-byte buffer

Specifying an offset within the limits of the byte buffer but writing an item that is larger than what fits results in the byte buffer being extended:

>>> buffer = bytearray(2)
>>> buffer
bytearray(b'\x00\x00')
>>>
>>> pack_into(UInt16, buffer, 1, 0xaabb)
>>> buffer
bytearray(b'\x00\xaa\xbb')

Obtaining Bytes Summary Dump

For every pack method/function shown in the previous sections, an “and_dump” variation exists to provide a bytes summary dump in addition to the packed item bytes. The following example shows one variation. But variations exist for all the plum type pack functions/methods.

>>> from plum import pack_and_dump
>>>
>>> fmt = {'first': UInt8, 'last': Sample}
>>> buffer, dump = pack_and_dump(fmt, first=0xaa, last=Sample(m1=1, m2=2))
>>> buffer
bytearray(b'\xaa\x00\x01\x02')
>>> print(dump)  # notice names in access column and offset reflects placement in buffer
+--------+-------------+-------+-------+--------+
| Offset | Access      | Value | Bytes | Type   |
+--------+-------------+-------+-------+--------+
| 0      | ['first']   | 170   | aa    | UInt8  |
|        | ['last']    |       |       | Sample |
| 1      |   [0] (.m1) | 1     | 00 01 | UInt16 |
| 3      |   [1] (.m2) | 2     | 02    | UInt8  |
+--------+-------------+-------+-------+--------+

Tip

The “and_dump” packing variations exist primarily for ease of use and performance. Otherwise a dump as a separate operation after an item is packed would redo identical work (and in some cases require extra work to instantiate the type to have access to the dump property).