Sized object to bytes and bytes to sized object transform.
[plum.sized] Module Reference¶
The plum.sized
module provides the SizedX
transform which converts
between a variably sized object of a particular type and a bytes sequence.
When packed, the generated bytes sequence contains the object size and the
object’s bytes. When unpacked, the object size located in the bytes sequence
is used to limit how many bytes that follow it are interpreted when unpacking
the object. This reference page demonstrates creating and using an SizedX
transform as well as provides API details.
The SizedX
transform accepts the following arguments:
fmt: object format size_fmt: size format int ratio: number of bytes per increment of size member (default=1) int offset: difference in size (in bytes, default=0) size_access: dump access description for size (default=”–size–“) name: transform name (for representations including dump format column)
The examples shown on this page require the following setup:
>>> from typing import List
>>> from plum.array import ArrayX
>>> from plum.bigendian import uint8, uint16
>>> from plum.sized import SizedX
>>> from plum.structure import Structure, member
>>> from plum.utilities import pack, unpack
Example¶
In this example, we’ll use an array transform that has no fixed dimension and instead transforms between byte sequences and lists of integers, either of which may be any size:
>>> unsized_array = ArrayX(fmt=uint16)
>>>
>>> unpack(fmt=unsized_array, buffer=b'\x00\x00\x00\x01\x00\x02')
[0, 1, 2]
>>>
>>> pack([0, 1, 2], fmt=unsized_array)
b'\x00\x00\x00\x01\x00\x02'
This type of transform becomes problematic during unpack operations when other data
follows the array since a dimensionless array transform consumes all remaining
bytes in a byte sequence. Structured data designs often contain a size indicator
just before the array data for the purpose of limiting the array. The SizedX
transform serves this use case:
>>> class Struct(Structure):
... array: List[int] = member(fmt=SizedX(size_fmt=uint8, fmt=unsized_array, ratio=2))
... bookend: int = member(fmt=uint8)
...
>>> bindata = b'\x03\x00\x00\x00\x01\x00\x02\x99'
>>>
>>> # unpack operation uses size header to limit array to data just before bookend value
>>> struct = Struct.unpack(bindata)
>>> struct.dump()
+--------+------------+-------+-------+--------------------+
| Offset | Access | Value | Bytes | Format |
+--------+------------+-------+-------+--------------------+
| | | | | Struct (Structure) |
| | array | | | sized: List[int] |
| 0 | --size-- | 3 | 03 | uint8 |
| | | | | List[int] |
| 1 | [0] | 0 | 00 00 | uint16 |
| 3 | [1] | 1 | 00 01 | uint16 |
| 5 | [2] | 2 | 00 02 | uint16 |
| 7 | bookend | 153 | 99 | uint8 |
+--------+------------+-------+-------+--------------------+
>>>
>>> # pack operation automatically inserts size header based on size of array data
>>> bindata, dump = Struct(array=[0, 1, 2], bookend=0x99).ipack_and_dump()
>>> dump()
+--------+------------+-------+-------+--------------------+
| Offset | Access | Value | Bytes | Format |
+--------+------------+-------+-------+--------------------+
| | | | | Struct (Structure) |
| | array | | | sized: List[int] |
| 0 | --size-- | 3 | 03 | uint8 |
| | | | | List[int] |
| 1 | [0] | 0 | 00 00 | uint16 |
| 3 | [1] | 1 | 00 01 | uint16 |
| 5 | [2] | 2 | 00 02 | uint16 |
| 7 | bookend | 153 | 99 | uint8 |
+--------+------------+-------+-------+--------------------+
>>> bindata
b'\x03\x00\x00\x00\x01\x00\x02\x99'
API Reference¶
-
class
plum.sized.
SizedX
(fmt: Union[plum.transform.Transform, Type[plum.data.Data]], size_fmt: plum.int.IntX, ratio: float = 1, offset: int = 0, size_access: str = '--size--', name: Optional[str] = None)¶ Sized object to bytes and bytes to sized object transform.
Parameters: - fmt – object format
- size_fmt – size format
- ratio (int) – number of bytes per increment of size member
- offset (int) – difference in size (in bytes)
- size_access – dump access description for size
-
fmt
¶ Sized object format.
-
name
¶ Transform format name (for repr and dump “Format” column).
-
nbytes
¶ Transform format size in bytes.
-
offset
¶ Difference in size (in bytes).
-
ratio
¶ Number of bytes per increment of size.
-
size_access
¶ Description to list in access column for size (e.g. “–size–“).
-
size_fmt
¶ Size format.
-
pack
(value: Any) → bytes¶ Pack value as formatted bytes.
Raises: PackError
if type error, value error, etc.
-
pack_and_dump
(value: Any) → Tuple[bytes, plum.dump.Dump]¶ Pack value as formatted bytes and produce bytes summary.
Raises: PackError
if type error, value error, etc.
-
unpack
(buffer: bytes) → Any¶ Unpack value from formatted bytes.
Raises: UnpackError
if insufficient bytes, excess bytes, or value error
-
unpack_and_dump
(buffer: bytes) → Tuple[Any, plum.dump.Dump]¶ Unpack value from bytes and produce packed bytes summary.
Raises: UnpackError
if insufficient bytes, excess bytes, or value error