Python marshal: Internal Serialization

The Python interpreter needs a fast way to store and load selected internal structures, especially code objects used in .pyc files. The Python marshal module exposes this low-level binary format for simple values and interpreter data. It can support specialized tools, but it is not designed as a general persistence, interchange, or external-input format.

This guide covers dump(), load(), dumps(), loads(), format versions, and blocking code objects. It complements our articles about pickle, pickletools, py_compile, bytecode with dis, and file comparison.

Why marshal exists

Marshal was created for Python’s internal needs. Its main purpose is helping the interpreter read and write internal structures, not providing a stable application protocol.

Unlike JSON, it does not prioritize interoperability. Unlike pickle, it does not attempt to serialize arbitrary class instances. The documentation explicitly warns that format details may change between versions.

Serialize in memory

marshal.dumps() converts a supported value into bytes.

import marshal

obj = {
    "name": "Ana",
    "scores": [10, 20, 30],
}

data = marshal.dumps(obj)
print(type(data), len(data))

The result is binary and should not be edited manually.

Restore from bytes

marshal.loads() reads one value from bytes.

restored = marshal.loads(data)
print(restored)

Use only bytes produced by a trusted and compatible environment. The official marshal documentation explicitly warns against untrusted data.

Write to a file

dump() writes a value to a binary file object.

with open("data.marshal", "wb") as file:
    marshal.dump(obj, file)

If the value contains an unsupported type, the function raises ValueError. Invalid partial data may already have been written, so the destination should not be reused after failure.

Atomic writing

Write to a temporary file and replace the destination only after success.

from pathlib import Path

path = Path("data.marshal")
temporary = path.with_suffix(".tmp")

with temporary.open("wb") as file:
    marshal.dump(obj, file)

temporary.replace(path)

This reduces corruption caused by serialization errors, although true crash durability may require flushing and filesystem synchronization.

Read from a file

load() reads one value from the file’s current position.

with open("data.marshal", "rb") as file:
    obj = marshal.load(file)

Bytes after the first value remain unread. This permits concatenating values, but the application needs a count, framing protocol, or another way to determine boundaries.

Several values in sequence

with open("sequence.marshal", "wb") as file:
    marshal.dump({"id": 1}, file)
    marshal.dump({"id": 2}, file)

with open("sequence.marshal", "rb") as file:
    first = marshal.load(file)
    second = marshal.load(file)

Without framing, it can be difficult to distinguish a valid end from truncation in a larger application protocol.

Supported types

Marshal handles several simple internal types, including:

• None, booleans, and numbers;
• strings and byte objects;
• tuples, lists, and dictionaries;
• sets and frozensets;
• selected additional structures depending on the format version;
• code objects when permitted.

Arbitrary user-defined instances are not supported as they are by pickle.

Recursive containers

Modern format versions can represent selected recursive containers.

items = []
items.append(items)

data = marshal.dumps(items)
restored = marshal.loads(data)
assert restored[0] is restored

Extremely deep structures can still hit implementation limits. Do not process unbounded nesting.

Format version

The module exposes marshal.version, the current default format version.

print(marshal.version)

data = marshal.dumps(obj, marshal.version)

The format argument does not guarantee complete compatibility among interpreter releases. A recognized format may still contain a type whose representation changed.

Format version is not Python version

The marshal format number is independent from the Python release number. Code-object structures can change even when the surrounding marshal format is understood.

Record implementation, complete Python version, platform, and format version when compatibility matters.

Code objects

Code objects contain bytecode, constants, names, and metadata for a compiled function or module.

code = compile("result = 2 + 2", "<example>", "exec")
data = marshal.dumps(code)
restored = marshal.loads(data)

Executing the restored object remains code execution.

environment = {}
exec(restored, environment)
print(environment["result"])

Never execute code objects from an unknown source.

Block code objects

Recent Python versions provide the keyword-only allow_code argument.

data = marshal.dumps(
    obj,
    allow_code=False,
)

restored = marshal.loads(
    data,
    allow_code=False,
)

When false, serializing or loading code objects is rejected. This removes one category of content but does not make hostile bytes safe.

Code-object compatibility

The documentation warns that code-object formats are not compatible across Python versions. Loading a code object for the wrong release has undefined behavior.

Do not store code objects as long-lived application caches. Use the official .pyc mechanism, which includes interpreter tags and validation headers.

marshal and pyc files

Although .pyc files use marshal for the code-object section, they also contain a header managed by importlib. Writing marshal.dumps(code) alone does not create a valid pyc.

Use py_compile or compileall to generate bytecode caches.

marshal versus pickle

Pickle supports custom classes and reconstruction protocols, but may invoke arbitrary callables during loading. Marshal supports fewer types and remains unsafe for untrusted data.

Neither format belongs at an upload boundary. Use JSON, a database, or a schema-driven format for external data.

marshal versus JSON

JSON offers interoperability, readable text, and a restricted data model. Marshal is Python-specific and optimized for interpreter details.

Even when marshal is smaller, lack of a long-term compatibility guarantee is usually a larger cost for application persistence.

Detect truncation

An incomplete file may raise EOFError, ValueError, or another reading error.

try:
    with open("data.marshal", "rb") as file:
        obj = marshal.load(file)
except (EOFError, ValueError, TypeError) as error:
    print("Invalid file:", error)

Do not attempt to use a partially read value.

External integrity checks

Internal data can be stored with a separate hash or HMAC.

import hashlib

digest = hashlib.sha256(data).hexdigest()

A plain hash detects accidental corruption but not an attacker who can replace both file and hash. Use keyed HMAC for authenticity.

Resource limits

Malformed data can attempt to create very large or deeply nested structures. Limit file size before loading and process potentially hostile input in a subprocess with memory and time restrictions.

from pathlib import Path

path = Path("data.marshal")
if path.stat().st_size > 10_000_000:
    raise ValueError("file is too large")

A size check is only one layer and does not replace isolation.

Audit events

Marshal operations raise Python auditing events. Embedded runtimes and security policies can observe loading and writing.

Do not rely on audit hooks alone to make unsafe input acceptable. Origin validation must happen before the marshal call.

Testing and tool usage

Marshal can be appropriate in interpreter tests, format experiments, and tools working with ephemeral artifacts from the same runtime.

Document that the file is disposable and should be regenerated after any interpreter change.

Round-trip testing

def test_round_trip():
    original = {
        "ids": [1, 2, 3],
        "active": True,
    }
    restored = marshal.loads(marshal.dumps(original))
    assert restored == original

Test supported types, empty input, truncation, the selected version, and allow_code=False.

Common mistakes

• Using marshal as a long-lived database.
• Loading bytes received from users.
• Assuming compatibility among Python versions.
• Executing restored code objects.
• Keeping a partially written file after ValueError.
• Confusing marshal output with a complete pyc file.
• Failing to limit size and depth.
• Treating allow_code=False as a sandbox.

Best practices

• Use marshal only for internal ephemeral needs.
• Record Python and format versions.
• Disable code objects when unnecessary.
• Write through a temporary file and replacement.
• Check size before loading.
• Authenticate internal artifacts when needed.
• Regenerate data after runtime changes.
• Choose stable formats for application records.

Conclusion

The Python marshal module provides fast binary serialization for a selected set of types and interpreter structures. It helps explain part of bytecode-cache implementation and can support specialized tooling.

Its contract is deliberately narrow: the format is not stable for general persistence, code objects are not portable, and unknown data is unsafe. Use marshal only in controlled environments with disposable artifacts, resource limits, and a clearly defined runtime compatibility policy.