Sized object to bytes and bytes to sized object transform.

[plum.sized] Module Reference

The plum.sized module provides the SizedX transform which converts between a variably sized object of a particular type and a bytes sequence. When packed, the generated bytes sequence contains the object size and the object’s bytes. When unpacked, the object size located in the bytes sequence is used to limit how many bytes that follow it are interpreted when unpacking the object. This reference page demonstrates creating and using an SizedX transform as well as provides API details.

The SizedX transform accepts the following arguments:

fmt:object format
size_fmt:size format
int ratio:number of bytes per increment of size member (default=1)
int offset:difference in size (in bytes, default=0)
size_access:dump access description for size (default=”–size–“)
name:transform name (for representations including dump format column)

The examples shown on this page require the following setup:

>>> from typing import List
>>> from plum.array import ArrayX
>>> from plum.bigendian import uint8, uint16
>>> from plum.sized import SizedX
>>> from plum.structure import Structure, member
>>> from plum.utilities import pack, unpack

Example

In this example, we’ll use an array transform that has no fixed dimension and instead transforms between byte sequences and lists of integers, either of which may be any size:

>>> unsized_array = ArrayX(fmt=uint16)
>>>
>>> unpack(fmt=unsized_array, buffer=b'\x00\x00\x00\x01\x00\x02')
[0, 1, 2]
>>>
>>> pack([0, 1, 2], fmt=unsized_array)
b'\x00\x00\x00\x01\x00\x02'

This type of transform becomes problematic during unpack operations when other data follows the array since a dimensionless array transform consumes all remaining bytes in a byte sequence. Structured data designs often contain a size indicator just before the array data for the purpose of limiting the array. The SizedX transform serves this use case:

>>> class Struct(Structure):
...     array: List[int] = member(fmt=SizedX(size_fmt=uint8, fmt=unsized_array, ratio=2))
...     bookend: int = member(fmt=uint8)
...
>>> bindata = b'\x03\x00\x00\x00\x01\x00\x02\x99'
>>>
>>> # unpack operation uses size header to limit array to data just before bookend value
>>> struct = Struct.unpack(bindata)
>>> struct.dump()
+--------+------------+-------+-------+--------------------+
| Offset | Access     | Value | Bytes | Format             |
+--------+------------+-------+-------+--------------------+
|        |            |       |       | Struct (Structure) |
|        | array      |       |       | sized: List[int]   |
| 0      |   --size-- | 3     | 03    | uint8              |
|        |            |       |       | List[int]          |
| 1      |   [0]      | 0     | 00 00 | uint16             |
| 3      |   [1]      | 1     | 00 01 | uint16             |
| 5      |   [2]      | 2     | 00 02 | uint16             |
| 7      | bookend    | 153   | 99    | uint8              |
+--------+------------+-------+-------+--------------------+
>>>
>>> # pack operation automatically inserts size header based on size of array data
>>> bindata, dump = Struct(array=[0, 1, 2], bookend=0x99).ipack_and_dump()
>>> dump()
+--------+------------+-------+-------+--------------------+
| Offset | Access     | Value | Bytes | Format             |
+--------+------------+-------+-------+--------------------+
|        |            |       |       | Struct (Structure) |
|        | array      |       |       | sized: List[int]   |
| 0      |   --size-- | 3     | 03    | uint8              |
|        |            |       |       | List[int]          |
| 1      |   [0]      | 0     | 00 00 | uint16             |
| 3      |   [1]      | 1     | 00 01 | uint16             |
| 5      |   [2]      | 2     | 00 02 | uint16             |
| 7      | bookend    | 153   | 99    | uint8              |
+--------+------------+-------+-------+--------------------+
>>> bindata
b'\x03\x00\x00\x00\x01\x00\x02\x99'

API Reference

class plum.sized.SizedX(fmt: Union[plum.transform.Transform, Type[plum.data.Data]], size_fmt: plum.int.IntX, ratio: float = 1, offset: int = 0, size_access: str = '--size--', name: Optional[str] = None)

Sized object to bytes and bytes to sized object transform.

Parameters:
  • fmt – object format
  • size_fmt – size format
  • ratio (int) – number of bytes per increment of size member
  • offset (int) – difference in size (in bytes)
  • size_access – dump access description for size
fmt

Sized object format.

name

Transform format name (for repr and dump “Format” column).

nbytes

Transform format size in bytes.

offset

Difference in size (in bytes).

ratio

Number of bytes per increment of size.

size_access

Description to list in access column for size (e.g. “–size–“).

size_fmt

Size format.

pack(value: Any) → bytes

Pack value as formatted bytes.

Raises:PackError if type error, value error, etc.
pack_and_dump(value: Any) → Tuple[bytes, plum.dump.Dump]

Pack value as formatted bytes and produce bytes summary.

Raises:PackError if type error, value error, etc.
unpack(buffer: bytes) → Any

Unpack value from formatted bytes.

Raises:UnpackError if insufficient bytes, excess bytes, or value error
unpack_and_dump(buffer: bytes) → Tuple[Any, plum.dump.Dump]

Unpack value from bytes and produce packed bytes summary.

Raises:UnpackError if insufficient bytes, excess bytes, or value error