Classes and utilities for packing/unpacking bytes.

[plum] Pack/Unpack Memory

Pipeline Status Coverage Report Documentation Status

The plum-py Python package provides classes and utility functions to transform byte sequences into Python objects and back. While similar in purpose to Python’s standard library struct module, this package provides a larger set of format specifiers and is extensible, allowing you to easily create complex ones of your own.

The following summarizes the plum transforms available in this package.

Transform Description
plum.array list of uniformly typed
plum.attrdict dictionary of uniquely typed items
plum.bigendian common big endian byte order transforms
plum.bitfields integer with bit field accessors
plum.bytes array of bytes
plum.decimal fixed point decimal numbers
plum.enum integer enumerated constants
plum.flag integer with bit flags
plum.float floating point
plum.int integers
plum.ipaddress IP address/network/interface objects
plum.items collection of uniquely typed items
plum.littleendian common little endian byte order transforms
plum.none no bytes
plum.optional optional item
plum.str strings
plum.sized variably sized object with size header
plum.structure structure of uniquely typed members

Quick Start

Numeric Types

Numeric transforms control the format of numeric data bytes. You may define your own transforms to choose your own naming conventions or simply use the common transforms provided by either the bigendian or ~plum.littleendian modules:

>>> from plum.littleendian import uint8, uint16, single
>>> from plum.utilities import pack, unpack
>>>
>>> fmt = [uint8, uint16, single]
>>>
>>> buffer = pack([2, 1, 0], fmt)
>>> buffer
b'\x02\x01\x00\x00\x00\x00\x00'
>>>
>>> unpack(fmt, buffer)
[2, 1, 0.0]

Arrays

Without dims, ArrayX transforms accept any number of items when packing and greedily consume bytes until exhaustion when unpacking:

>>> from plum.array import ArrayX
>>>
>>> greedy_array = ArrayX(fmt=uint8)
>>>
>>> # greedy arrays accept any number of items when packing
>>> pack([1, 2], fmt=greedy_array)
b'\x01\x02'
>>>
>>> # greedy arrays consume everything that is left in the buffer
>>> unpack(greedy_array, b'\x01\x02\x03\x04')
[1, 2, 3, 4]

Specifying dims controls the the array dimensions:

>>> array_2x2 = ArrayX(fmt=uint8, dims=(2, 2))
>>>
>>> buffer = array_2x2.pack([[1, 2], [3, 4]])
>>> buffer
b'\x01\x02\x03\x04'
>>>
>>> array_2x2.unpack(buffer)
[[1, 2], [3, 4]]

Structures

Structure offers a succinct and intuitive method for defining the bytes format for a sequence of uniquely typed members:

>>> from plum.structure import Structure, member
>>>
>>> class MyStruct(Structure):
...     byte: int = member(fmt=uint8)
...     word: int = member(fmt=uint16, default=0)
...     array: list = member(fmt=array_2x2)
...
>>> buffer = pack(MyStruct(byte=2, array=[[1, 2], [3, 4]]))
>>> buffer
b'\x02\x00\x00\x01\x02\x03\x04'
>>>
>>> mystruct = unpack(MyStruct, buffer)
>>> mystruct
MyStruct(byte=2, word=0, array=[[1, 2], [3, 4]])
>>>
>>> # access members via attribute or index
>>> mystruct.byte
2
>>> mystruct[0]
2

Structure offers many features. Be sure to consult the API reference and Tutorials since you will likely use this type often.

Byte Breakdown Summaries

Variants to the pack() and unpack() utility functions exist for additionally obtaining a byte by byte summary of the transformation:

>>> from plum.utilities import pack_and_dump, unpack_and_dump
>>>
>>> buffer, dump = pack_and_dump([[0, 1], [2, 3]], fmt=array_2x2)
>>> print(dump)
+--------+--------+-------+-------+-----------------+
| Offset | Access | Value | Bytes | Format          |
+--------+--------+-------+-------+-----------------+
|        |        |       |       | List[List[int]] |
|        | [0]    |       |       |                 |
| 0      |   [0]  | 0     | 00    | uint8           |
| 1      |   [1]  | 1     | 01    | uint8           |
|        | [1]    |       |       |                 |
| 2      |   [0]  | 2     | 02    | uint8           |
| 3      |   [1]  | 3     | 03    | uint8           |
+--------+--------+-------+-------+-----------------+
>>> array, dump = unpack_and_dump(array_2x2, buffer)
>>> print(dump)
+--------+--------+-------+-------+-----------------+
| Offset | Access | Value | Bytes | Format          |
+--------+--------+-------+-------+-----------------+
|        |        |       |       | List[List[int]] |
|        | [0]    |       |       |                 |
| 0      |   [0]  | 0     | 00    | uint8           |
| 1      |   [1]  | 1     | 01    | uint8           |
|        | [1]    |       |       |                 |
| 2      |   [0]  | 2     | 02    | uint8           |
| 3      |   [1]  | 3     | 03    | uint8           |
+--------+--------+-------+-------+-----------------+
>>> array
[[0, 1], [2, 3]]

Structure instances offer a dump property which you may convert to a string and print yourself. Alternatively, it prints the summary when executed:

>>> mystruct = MyStruct(byte=2, array=[[1, 2], [3, 4]])
>>> mystruct.dump()
+--------+---------+-------+-------+----------------------+
| Offset | Access  | Value | Bytes | Format               |
+--------+---------+-------+-------+----------------------+
|        |         |       |       | MyStruct (Structure) |
| 0      | byte    | 2     | 02    | uint8                |
| 1      | word    | 0     | 00 00 | uint16               |
|        | array   |       |       | List[List[int]]      |
|        |   [0]   |       |       |                      |
| 3      |     [0] | 1     | 01    | uint8                |
| 4      |     [1] | 2     | 02    | uint8                |
|        |   [1]   |       |       |                      |
| 5      |     [0] | 3     | 03    | uint8                |
| 6      |     [1] | 4     | 04    | uint8                |
+--------+---------+-------+-------+----------------------+

Enumerations

Enumeration transforms control the format of integer enumeration data bytes.

>>> from enum import IntEnum
>>> from plum.enum import EnumX
>>>
>>> class Pet(IntEnum):
...     CAT = 0
...     DOG = 1
...
>>> class Family(Structure):
...     nkids: int = member(fmt=uint8)
...     pet: Pet = member(fmt=EnumX(enum=Pet, nbytes=1))
...
>>> family = Family.unpack(b'\x02\x01')
>>> family.dump()
+--------+--------+---------+-------+--------------------+
| Offset | Access | Value   | Bytes | Format             |
+--------+--------+---------+-------+--------------------+
|        |        |         |       | Family (Structure) |
| 0      | nkids  | 2       | 02    | uint8              |
| 1      | pet    | Pet.DOG | 01    | Pet (IntEnum)      |
+--------+--------+---------+-------+--------------------+
>>>
>>> family.pet
<Pet.DOG: 1>

The default EnumX behavior raises an exception when packing or unpacking values that are not members of the enumeration. See the EnumX API Reference to discover how to create an enum transformation which allows packing and unpacking operations to succeed for values that aren’t members of the enumeration.