[plum] Tutorial: Arbitrary Formats¶
The format fmt
specifier argument for the pack()
and unpack()
utility functions offer tremendous flexibility. This tutorial details
the various possible constructs and their behavior.
Transform¶
When a transform is specified for fmt
, packing uses it to produce bytes
from the value provided:
>>> from plum.bigendian import uint8
>>> from plum.utilities import pack, unpack
>>>
>>> fmt = uint8
>>>
>>> buffer = pack(0, fmt)
>>> buffer
b'\x00'
Unpacking produces a Python object per the transform specified as fmt
:
>>> unpack(fmt, buffer)
0
List¶
When a list of transforms are specified for fmt
, packing uses them to
produce bytes from the list (or any iterable) of values provided:
>>> fmt = [uint8, uint8, uint8]
>>> buffer = pack([0, 1, 2], fmt)
>>> buffer
b'\x00\x01\x02'
Unpacking produces a list of Python objects per the transforms specified:
>>> unpack(fmt, buffer)
[0, 1, 2]
Tuple¶
Providing a tuple of transforms for the fmt
works exactly like providing a
list except unpacking produces a tuple of Python objects instead of a list of
Python objects:
>>> fmt = (uint8, uint8, uint8)
>>> buffer = pack([0, 1, 2], fmt)
>>> buffer
b'\x00\x01\x02'
>>> unpack(fmt, buffer)
(0, 1, 2)
Dict¶
Providing a dictionary of transforms works similarly except when packing operations expect a dictionary of values and unpacking produces a dictionary of Python objects:
>>> fmt = {"a": uint8, "b": uint8}
>>> buffer = pack({"a": 0, "b": 1}, fmt)
>>> buffer
b'\x00\x01'
>>> unpack(fmt, buffer)
{'a': 0, 'b': 1}
Nesting¶
pack()
and unpack()
accept arbitrarily nested dictionaries, lists,
and tuples of transforms for fmt
. For example:
>>> fmt = [(uint8, uint8), {"a": uint8, "b": uint8}]
>>> buffer = pack([(0, 1), {"a": 2, "b": 3}], fmt)
>>> buffer
b'\x00\x01\x02\x03'
>>> unpack(fmt, buffer)
[(0, 1), {'a': 2, 'b': 3}]
None¶
The pack()
utility function supports None
as the fmt
for
applications where the value is a data store with transform properties
(e.g. BitFields
, Structure
):
>>> from plum.structure import Structure, member
>>>
>>> class MyStruct(Structure):
... a: int = member(fmt=uint8)
... b: int = member(fmt=uint8)
>>>
>>> # value is a data store with transform properties
>>> pack(MyStruct(a=1, b=2), fmt=None)
b'\x01\x02'
>>>
>>> # fmt defaults to None, no need to specify it!
>>> pack(MyStruct(a=1, b=2))
b'\x01\x02'
If the value is already bytes
, pack()
passes them through:
>>> pack(b'\x00\x01', fmt=None)
b'\x00\x01'
Similarly, pack()
supports None
within any dictionary, list, or
tuple of transforms used as the fmt
:
>>> fmt = [uint8, None]
>>> pack([0, MyStruct(a=1, b=2)], fmt)
b'\x00\x01\x02'
>>> pack([0, b'\x01\x02'], fmt)
b'\x00\x01\x02'
In places where None
was specified, pack()
also accepts a
(value, fmt) tuple pair:
>>> pack([(0, uint8), (1, uint8)], fmt=[None, None])
b'\x00\x01'
Where ever fmt
is None
, pack()
accepts dictionaries, lists, and
tuples of (value, fmt) pairs for the value allowing complex structures to be
accepted:
>>> # explicit fmt=None
>>> pack([(0, uint8), {"a": (1, uint8), "b": (2, uint8)}], fmt=None)
b'\x00\x01\x02'
>>>
>>> # implicit fmt=None
>>> pack([(0, uint8), {"a": (1, uint8), "b": (2, uint8)}])
b'\x00\x01\x02'
>>>
>>> # mix
>>> pack([0, {"a": (1, uint8), "b": (2, uint8)}], fmt=[uint8, None])
b'\x00\x01\x02'
When fmt
is None
, the unpack()
utility function returns the
remaining available bytes:
>>> unpack(fmt=None, buffer=b'\x00\x01')
b'\x00\x01'
>>> unpack(fmt=[uint8, None], buffer=b'\x00\x01\x02')
[0, b'\x01\x02']