Array to bytes and bytes to array transform.
[plum.array] Module Reference¶
The plum.array
module provides the ArrayX
transform which converts
lists of uniformly typed items into bytes and bytes into lists of uniformly
typed items. This reference page demonstrates creating and using an ArrayX
transform as well as provides API details.
The examples shown on this page require the following setup:
>>> from plum.array import ArrayX
>>> from plum.bigendian import uint8, uint16
>>> from plum.utilities import pack_and_dump, unpack
Dimensioned Array¶
The ArrayX
transform accepts the following arguments:
fmt: format of each array element dims: length of each array dimension name: transform name (for representations including dump format column)
The ArrayX
transform fmt
argument accepts a transform that describes
the conversion between bytes and the array’s members.
The ArrayX
transform dims
argument accepts a tuple of integers that describe
the length of each dimension of the array list.
>>> array_2x2 = ArrayX(fmt=uint16, dims=(2, 2))
>>> array_2x2
<transform 'List[List[int]]'>
When packing, the transform accepts any iterable (list, tuple, etc.) and ensures the
iterable has the proper dimensions (and raises a PackError
if not):
>>> array_2x2.pack([[1, 2], [3, 4]])
b'\x00\x01\x00\x02\x00\x03\x00\x04'
>>>
>>> array_2x2.pack([[1, 2], [3]])
Traceback (most recent call last):
...
plum.exceptions.PackError:
<BLANKLINE>
+--------+--------+-----------+-------+-----------------+
| Offset | Access | Value | Bytes | Format |
+--------+--------+-----------+-------+-----------------+
| | | | | List[List[int]] |
| | [0] | | | |
| 0 | [0] | 1 | 00 01 | uint16 |
| 2 | [1] | 2 | 00 02 | uint16 |
| | [1] | | | |
| 4 | [0] | 3 | 00 03 | uint16 |
| | [1] | <missing> | | |
+--------+--------+-----------+-------+-----------------+
<BLANKLINE>
TypeError occurred during pack operation:
<BLANKLINE>
invalid value, expected iterable of 2 length, got iterable of length 1
When unpacking, the transform produces a list of items from the unpacked bytes and uses
the dims
to control the size of the list for each dimension. If too few or too many
bytes are provided, the array transform raises an UnpackError
.
>>> buffer = b'\x00\x01\x00\x02\x00\x03\x00\x04'
>>>
>>> array_2x2.unpack(buffer)
[[1, 2], [3, 4]]
>>>
>>> array_2x2.unpack(buffer[:-1])
Traceback (most recent call last):
...
plum.exceptions.UnpackError:
<BLANKLINE>
+--------+--------+----------------------+-------+-----------------+
| Offset | Access | Value | Bytes | Format |
+--------+--------+----------------------+-------+-----------------+
| | | | | List[List[int]] |
| | [0] | | | |
| 0 | [0] | 1 | 00 01 | uint16 |
| 2 | [1] | 2 | 00 02 | uint16 |
| | [1] | | | |
| 4 | [0] | 3 | 00 03 | uint16 |
| 6 | [1] | <insufficient bytes> | 00 | uint16 |
+--------+--------+----------------------+-------+-----------------+
<BLANKLINE>
InsufficientMemoryError occurred during unpack operation:
<BLANKLINE>
1 too few bytes to unpack uint16, 2 needed, only 1 available
Greedy Array¶
When the dims
argument is left to default to (None, )
, the resulting
transform becomes “greedy”. When packing, for any dimension specified as None
,
the transform accepts any number of items from the iterable (e.g. list, tuple, etc.)
and packs each into a bytes buffer per the specified fmt
.
>>> greedy_array = ArrayX(fmt=uint8)
>>> greedy_array
<transform 'List[int]'>
>>>
>>> buffer, dump = greedy_array.pack_and_dump([1, 2, 3])
>>> buffer
b'\x01\x02\x03'
>>> print(dump)
+--------+--------+-------+-------+-----------+
| Offset | Access | Value | Bytes | Format |
+--------+--------+-------+-------+-----------+
| | | | | List[int] |
| 0 | [0] | 1 | 01 | uint8 |
| 1 | [1] | 2 | 02 | uint8 |
| 2 | [2] | 3 | 03 | uint8 |
+--------+--------+-------+-------+-----------+
When unpacking, the transform consumes all remaining bytes in the buffer and produces a list of as many unpacked items as possible:
>>> array, dump = greedy_array.unpack_and_dump(b'\x01\x02\x03')
>>> array
[1, 2, 3]
>>> print(dump)
+--------+--------+-------+-------+-----------+
| Offset | Access | Value | Bytes | Format |
+--------+--------+-------+-------+-----------+
| | | | | List[int] |
| 0 | [0] | 1 | 01 | uint8 |
| 1 | [1] | 2 | 02 | uint8 |
| 2 | [2] | 3 | 03 | uint8 |
+--------+--------+-------+-------+-----------+
Note
The ArrayX
transform accepts None
for any of the dimensions in multi-dimensional
applications, but limitations and special behaviors exist. For packing, for any
dimension specified as None
, the transform accepts any number of items for that
particular dimension and no consistency checks are enforced. For unpacking, the transform
only allows None
for the outermost (first) dimension, otherwise a :class:UnpackError
results.
Sized Arrays¶
The ArrayX
transform constructor accepts an IntX
transform
instance. When unpacking, the transform first unpacks the dimension of the
array from the byte sequence, then unpacks the array data accordingly.
When packing, the transform includes the dimensions of the array data in
the generated byte sequence. For example:
>>> single_dim_array = ArrayX(fmt=uint8, dims=uint16)
>>> bindata, dump = pack_and_dump([1, 2], fmt=single_dim_array)
>>> dump()
+--------+--------+-------+-------+-----------+
| Offset | Access | Value | Bytes | Format |
+--------+--------+-------+-------+-----------+
| | | | | List[int] |
| 0 | len() | 2 | 00 02 | uint16 |
| 2 | [0] | 1 | 01 | uint8 |
| 3 | [1] | 2 | 02 | uint8 |
+--------+--------+-------+-------+-----------+
>>>
>>> unpack(single_dim_array, bindata)
[1, 2]
Similarly, the ArrayX
transform constructor also accepts an ArrayX
transform instance for multiple dimensions:
>>> multi_dim_array = ArrayX(fmt=uint8, dims=ArrayX(fmt=uint8, dims=[2]))
>>> bindata, dump = pack_and_dump([[1, 2, 3], [4, 5, 6]], fmt=multi_dim_array)
>>> dump()
+--------+----------+-------+-------+-----------------+
| Offset | Access | Value | Bytes | Format |
+--------+----------+-------+-------+-----------------+
| | | | | List[List[int]] |
| | --dims-- | | | List[int] |
| 0 | [0] | 2 | 02 | uint8 |
| 1 | [1] | 3 | 03 | uint8 |
| | [0] | | | |
| 2 | [0] | 1 | 01 | uint8 |
| 3 | [1] | 2 | 02 | uint8 |
| 4 | [2] | 3 | 03 | uint8 |
| | [1] | | | |
| 5 | [0] | 4 | 04 | uint8 |
| 6 | [1] | 5 | 05 | uint8 |
| 7 | [2] | 6 | 06 | uint8 |
+--------+----------+-------+-------+-----------------+
>>>
>>> unpack(multi_dim_array, bindata)
[[1, 2, 3], [4, 5, 6]]
Sized Arrays in Structures¶
See the Dimensioned Array Structure Member tutorial for information on creating structures with an array member and a second member for controlling its size (dimensions).
API Reference¶
-
class
plum.array.
ArrayX
(fmt: Union[plum.transform.Transform, Type[plum.data.Data]], dims: Union[Sequence[Optional[int]], ArrayX, plum.int.IntX, None] = None, name: Optional[str] = None)¶ Array to bytes and bytes to array transform.
-
dims
¶ Array dimensions.
-
fmt
¶ Array element format.
-
name
¶ Transform format name (for repr and dump “Format” column).
-
nbytes
¶ Transform format size in bytes.
-
pack
(value: Any) → bytes¶ Pack value as formatted bytes.
Raises: PackError
if type error, value error, etc.
-
pack_and_dump
(value: Any) → Tuple[bytes, plum.dump.Dump]¶ Pack value as formatted bytes and produce bytes summary.
Raises: PackError
if type error, value error, etc.
-
unpack
(buffer: bytes) → Any¶ Unpack value from formatted bytes.
Raises: UnpackError
if insufficient bytes, excess bytes, or value error
-
unpack_and_dump
(buffer: bytes) → Tuple[Any, plum.dump.Dump]¶ Unpack value from bytes and produce packed bytes summary.
Raises: UnpackError
if insufficient bytes, excess bytes, or value error
-