Classes and utilities for packing/unpacking bytes.

[plum] Pack/Unpack Memory

Pipeline Status Coverage Report Documentation Status

The plum-py Python package provides classes and utility functions to transform byte sequences into Python objects and back. While similar in purpose to Python’s standard library struct module, this package provides a larger set of format specifiers and is extensible, allowing you to easily create complex ones of your own.

The following summarizes the plum transforms available in this package.

Transform

Description

plum.array

list of uniformly typed

plum.attrdict

dictionary of uniquely typed items

plum.bigendian

common big endian byte order transforms

plum.bitfields

integer with bit field accessors

plum.bytes

array of bytes

plum.decimal

fixed point decimal numbers

plum.enum

integer enumerated constants

plum.flag

integer with bit flags

plum.float

floating point

plum.int

integers

plum.ipaddress

IP address/network/interface objects

plum.items

collection of uniquely typed items

plum.littleendian

common little endian byte order transforms

plum.none

no bytes

plum.optional

optional item

plum.str

strings

plum.sized

variably sized object with size header

plum.structure

structure of uniquely typed members

Quick Start

Numeric Types

Numeric transforms control the format of numeric data bytes. You may define your own transforms to choose your own naming conventions or simply use the common transforms provided by either the bigendian or ~plum.littleendian modules:

>>> from plum.littleendian import uint8, uint16, single
>>> from plum.utilities import pack, unpack
>>>
>>> fmt = [uint8, uint16, single]
>>>
>>> buffer = pack([2, 1, 0], fmt)
>>> buffer
b'\x02\x01\x00\x00\x00\x00\x00'
>>>
>>> unpack(fmt, buffer)
[2, 1, 0.0]

Arrays

Without dims, ArrayX transforms accept any number of items when packing and greedily consume bytes until exhaustion when unpacking:

>>> from plum.array import ArrayX
>>>
>>> greedy_array = ArrayX(fmt=uint8)
>>>
>>> # greedy arrays accept any number of items when packing
>>> pack([1, 2], fmt=greedy_array)
b'\x01\x02'
>>>
>>> # greedy arrays consume everything that is left in the buffer
>>> unpack(greedy_array, b'\x01\x02\x03\x04')
[1, 2, 3, 4]

Specifying dims controls the the array dimensions:

>>> array_2x2 = ArrayX(fmt=uint8, dims=(2, 2))
>>>
>>> buffer = array_2x2.pack([[1, 2], [3, 4]])
>>> buffer
b'\x01\x02\x03\x04'
>>>
>>> array_2x2.unpack(buffer)
[[1, 2], [3, 4]]

Structures

Structure offers a succinct and intuitive method for defining the bytes format for a sequence of uniquely typed members:

>>> from plum.structure import Structure, member
>>>
>>> class MyStruct(Structure):
...     byte: int = member(fmt=uint8)
...     word: int = member(fmt=uint16, default=0)
...     array: list = member(fmt=array_2x2)
...
>>> buffer = pack(MyStruct(byte=2, array=[[1, 2], [3, 4]]))
>>> buffer
b'\x02\x00\x00\x01\x02\x03\x04'
>>>
>>> mystruct = unpack(MyStruct, buffer)
>>> mystruct
MyStruct(byte=2, word=0, array=[[1, 2], [3, 4]])
>>>
>>> # access members via attribute or index
>>> mystruct.byte
2
>>> mystruct[0]
2

Structure offers many features. Be sure to consult the API reference and Tutorials since you will likely use this type often.

Byte Breakdown Summaries

Variants to the pack() and unpack() utility functions exist for additionally obtaining a byte by byte summary of the transformation:

>>> from plum.utilities import pack_and_dump, unpack_and_dump
>>>
>>> buffer, dump = pack_and_dump([[0, 1], [2, 3]], fmt=array_2x2)
>>> print(dump)
+--------+--------+-------+-------+-----------------+
| Offset | Access | Value | Bytes | Format          |
+--------+--------+-------+-------+-----------------+
|        |        |       |       | List[List[int]] |
|        | [0]    |       |       |                 |
| 0      |   [0]  | 0     | 00    | uint8           |
| 1      |   [1]  | 1     | 01    | uint8           |
|        | [1]    |       |       |                 |
| 2      |   [0]  | 2     | 02    | uint8           |
| 3      |   [1]  | 3     | 03    | uint8           |
+--------+--------+-------+-------+-----------------+
>>> array, dump = unpack_and_dump(array_2x2, buffer)
>>> print(dump)
+--------+--------+-------+-------+-----------------+
| Offset | Access | Value | Bytes | Format          |
+--------+--------+-------+-------+-----------------+
|        |        |       |       | List[List[int]] |
|        | [0]    |       |       |                 |
| 0      |   [0]  | 0     | 00    | uint8           |
| 1      |   [1]  | 1     | 01    | uint8           |
|        | [1]    |       |       |                 |
| 2      |   [0]  | 2     | 02    | uint8           |
| 3      |   [1]  | 3     | 03    | uint8           |
+--------+--------+-------+-------+-----------------+
>>> array
[[0, 1], [2, 3]]

Structure instances offer a dump property which you may convert to a string and print yourself. Alternatively, it prints the summary when executed:

>>> mystruct = MyStruct(byte=2, array=[[1, 2], [3, 4]])
>>> mystruct.dump()
+--------+---------+-------+-------+----------------------+
| Offset | Access  | Value | Bytes | Format               |
+--------+---------+-------+-------+----------------------+
|        |         |       |       | MyStruct (Structure) |
| 0      | byte    | 2     | 02    | uint8                |
| 1      | word    | 0     | 00 00 | uint16               |
|        | array   |       |       | List[List[int]]      |
|        |   [0]   |       |       |                      |
| 3      |     [0] | 1     | 01    | uint8                |
| 4      |     [1] | 2     | 02    | uint8                |
|        |   [1]   |       |       |                      |
| 5      |     [0] | 3     | 03    | uint8                |
| 6      |     [1] | 4     | 04    | uint8                |
+--------+---------+-------+-------+----------------------+

Enumerations

Enumeration transforms control the format of integer enumeration data bytes.

>>> from enum import IntEnum
>>> from plum.enum import EnumX
>>>
>>> class Pet(IntEnum):
...     CAT = 0
...     DOG = 1
...
>>> class Family(Structure):
...     nkids: int = member(fmt=uint8)
...     pet: Pet = member(fmt=EnumX(enum=Pet, nbytes=1))
...
>>> family = Family.unpack(b'\x02\x01')
>>> family.dump()
+--------+--------+---------+-------+--------------------+
| Offset | Access | Value   | Bytes | Format             |
+--------+--------+---------+-------+--------------------+
|        |        |         |       | Family (Structure) |
| 0      | nkids  | 2       | 02    | uint8              |
| 1      | pet    | Pet.DOG | 01    | Pet (IntEnum)      |
+--------+--------+---------+-------+--------------------+
>>>
>>> family.pet
<Pet.DOG: 1>

The default EnumX behavior raises an exception when packing or unpacking values that are not members of the enumeration. See the EnumX API Reference to discover how to create an enum transformation which allows packing and unpacking operations to succeed for values that aren’t members of the enumeration.