Python struct: Pack Binary Data

The Python struct module converts integers, floating-point values, booleans, and byte sequences into compact binary layouts. It also performs the reverse operation, interpreting bytes received from files, devices, or network connections according to a format string.

This is useful for protocols, file headers, C interoperability, sensors, and legacy formats. The main pitfall is relying on the host platform. Without a prefix, struct uses native byte order, native C sizes, and native alignment. For external interchange, define byte order and sizes explicitly.

Pack and unpack a record

import struct

format_string = ">Ih"
packet = struct.pack(format_string, 1_000_000, -12)
identifier, temperature = struct.unpack(format_string, packet)

print(packet.hex())
print(identifier, temperature)

> selects big-endian with standard sizes. I is an unsigned 32-bit integer and h is a signed 16-bit integer. unpack() always returns a tuple.

Byte order prefixes

• <: little-endian, standard sizes, no implicit alignment.
• >: big-endian, standard sizes.
• !: network order, equivalent to big-endian.
• =: native byte order with standard sizes.
• @: fully native order, size, and alignment.

import struct

print(struct.pack(">H", 1023).hex())  # 03ff
print(struct.pack("<H", 1023).hex())  # ff03

For files and protocols, avoid the implicit native mode. Select <, >, or ! so the layout is identical on every architecture.

Calculate required size

import struct

HEADER = struct.Struct("!4sBBHI")
print(HEADER.size)

with open("message.bin", "rb") as file:
    raw = file.read(HEADER.size)
    if len(raw) != HEADER.size:
        raise ValueError("incomplete header")
    magic, version, flags, kind, length = HEADER.unpack(raw)

unpack() needs an exact-size buffer. unpack_from() requires at least the format size after its offset. Validate the length before parsing.

Reuse a compiled Struct

A Struct object compiles the format once and exposes a reusable size and methods:

import struct

RECORD = struct.Struct("<Iff?")
payload = RECORD.pack(42, 18.5, 70.25, True)
identifier, x, y, active = RECORD.unpack(payload)

The module caches recent formats, but named objects make protocol code easier to document and audit.

Common format characters

• b/B: signed or unsigned 8-bit integer.
• h/H: 16-bit integer.
• i/I: standard 32-bit integer.
• q/Q: 64-bit integer.
• e, f, d: 16-, 32-, and 64-bit floating point.
• ?: boolean.
• s: fixed-length bytes.
• x: explicit padding byte.

Python 3.14 added F and D for single- and double-precision complex numbers.

Fixed-length byte strings

import struct

FORMAT = struct.Struct("!10sI")
raw = FORMAT.pack(b"sensor-1", 123)
name, reading = FORMAT.unpack(raw)
name = name.rstrip(b"\x00").decode("ascii")

10s is one ten-byte field. Longer input is truncated and shorter input is padded with zeros. Validate length before packing if truncation would be data loss.

Write into existing buffers

import struct

buffer = bytearray(1024)
HEADER = struct.Struct("!IHH")
HEADER.pack_into(buffer, 0, 900, 2, 7)
identifier, version, flags = HEADER.unpack_from(buffer, 0)

These methods accept buffer-protocol objects such as bytearray and memoryview, reducing temporary allocations in high-throughput binary pipelines.

Decode repeated records

import struct

RECORD = struct.Struct("<Ih")
data = b"".join([
    RECORD.pack(1, 20),
    RECORD.pack(2, 25),
    RECORD.pack(3, 18),
])

for identifier, value in RECORD.iter_unpack(data):
    print(identifier, value)

The total length must be a multiple of the record size. Reject trailing bytes because they may indicate corruption or a different protocol version.

Length-prefixed protocols

import struct

HEADER = struct.Struct("!4sBI")
MAX_PAYLOAD = 10 * 1024 * 1024

header = receive_exactly(HEADER.size)
magic, version, length = HEADER.unpack(header)

if magic != b"APP1" or version != 1:
    raise ValueError("unsupported protocol")
if length > MAX_PAYLOAD:
    raise ValueError("payload exceeds the limit")

payload = receive_exactly(length)

Never allocate solely from a peer-supplied length. Apply a maximum, timeout, rate limit, and message-count limit.

Networking and compressed bodies

Use ! for network byte order. A body may be compressed with Python zlib, but the header should clearly identify version, flags, algorithm, compressed length, and expanded limit. Internal VM values such as those described in Python opcode are not stable protocol identifiers.

Native alignment

With @, the platform C compiler may insert padding between fields. This is appropriate for mirroring a C struct in the same environment, but unsafe for portable storage.

import struct

print(struct.calcsize("@ci"))
print(struct.calcsize("@ic"))
print(struct.calcsize("=ci"))

Field order can change size in native mode. Standard modes add padding only when you explicitly request x.

Ranges and struct.error

import struct

try:
    struct.pack("!h", 100_000)
except struct.error as exc:
    raise ValueError("value does not fit a signed 16-bit field") from exc

Validate reserved values, flag combinations, NaN, infinity, and application-specific ranges as well.

Do not use struct as a universal serializer

struct does not store field names, schema evolution, or semantic types. It excels at compact, stable layouts. JSON, databases, or schema-based formats are better for flexible application objects.

Security checklist

• Validate buffer length before parsing.
• Define endianness explicitly.
• Bound declared payload lengths.
• Reject unsupported versions and codes.
• Use network timeouts.
• Define reserved and padding bytes.
• Never use pointer format P for external data.
• Fuzz binary parsers.

Testing strategy

Test minimum and maximum field values, swapped endianness, short buffers, extra bytes, incomplete records, NaN, infinity, NUL characters, future versions, and payloads above limits. Assert exact byte strings, not only round trips.

def test_header_bytes():
    assert HEADER.pack(b"APP1", 1, 5) == b"APP1\x01\x00\x00\x00\x05"

Best practices

• Create named Struct constants.
• Document every field and unit.
• Include magic bytes and a version.
• Use standard sizes for interchange.
• Validate before allocating.
• Use pack_into and memoryview when copies matter.
• Log errors without dumping sensitive binary content.

Conclusion

Python struct is an efficient bridge between Python values and binary layouts. It works well for headers, files, and protocols when byte order, size, versioning, and limits are explicit.

Read the official struct documentation and the buffer protocol documentation. For environment-based limits, see Python configparser, and for runtime flow use Python trace.