Python py_compile: Generate pyc Files

When a Python module is imported, the interpreter may cache its bytecode in a .pyc file inside __pycache__. The Python py_compile module generates this cache in advance, validates source during builds, and controls how the interpreter decides whether cached bytecode still matches the source.

This guide covers py_compile.compile(), PyCompileError, optimization levels, and timestamp or hash invalidation. It complements our articles about bytecode with dis, compileall, tracebacks, tokenize, and interactive compilation.

What a pyc file is

A .pyc file contains serialized bytecode and a header with cache-validation information. It can avoid recompiling unchanged source during selected startups, but it does not turn Python into a standalone native binary.

Bytecode remains implementation- and version-specific. A file produced for one CPython cache tag should not be assumed compatible with another.

Compile a source file

The main function accepts a source path.

import py_compile

path = py_compile.compile("my_module.py")
print(path)

By default, the output follows PEP 3147 and PEP 488 conventions, usually under __pycache__ with the interpreter tag.

Validate syntax during a build

Precompilation provides a simple way to detect SyntaxError before deployment.

from pathlib import Path
import py_compile

for file in Path("src").rglob("*.py"):
    py_compile.compile(
        str(file),
        doraise=True,
    )

For entire trees, compileall is more convenient. py_compile is appropriate when the application already has an exact file list.

doraise and PyCompileError

By default, a compilation error is written to stderr and the function returns None. With doraise=True, it raises PyCompileError.

try:
    py_compile.compile(
        "broken.py",
        doraise=True,
    )
except py_compile.PyCompileError as error:
    print("Compilation failed:", error)

Applications and pipelines should prefer the exception so they can control status and produce structured reports.

Choose the destination with cfile

cfile explicitly selects the output file.

py_compile.compile(
    "module.py",
    cfile="build/module.pyc",
    doraise=True,
)

Create the destination directory first. Custom paths can help packaging, while default conventions simplify importing and coexistence among Python versions.

Symlink protection

If the calculated destination is a symbolic link or non-regular file, the function raises FileExistsError.

The official py_compile documentation explains that writing follows importlib semantics, using a temporary file and rename to reduce partial results during concurrent writes. The check avoids silently replacing special objects.

Source names in tracebacks

dfile sets the source filename stored for tracebacks and related messages.

py_compile.compile(
    "src/package/module.py",
    dfile="/app/package/module.py",
    doraise=True,
)

This is useful when build paths differ from deployment paths. Use a meaningful location without exposing private directories.

Optimization level

optimize is passed to the built-in compile(). The default -1 selects the current interpreter’s optimization level.

py_compile.compile(
    "module.py",
    optimize=1,
    doraise=True,
)

Level 1 removes assertions and changes __debug__. Level 2 also removes many docstrings. Do not use optimization to hide source and do not assume a meaningful speed improvement without measurement.

Separate caches for optimization

When using the default destination, the cache tag includes the optimization variant. Different levels can coexist in __pycache__.

When selecting cfile manually, avoid overwriting variants that must remain separate. Record Python version, implementation, and optimization in the build process.

Invalidation modes

PycInvalidationMode determines how Python checks whether a .pyc is current.

• TIMESTAMP: compares source timestamp and size;
• CHECKED_HASH: stores a source hash and validates it at import;
• UNCHECKED_HASH: stores a hash but trusts an external build system to keep the cache current.

The selected mode is stored in the pyc header.

Timestamp invalidation

from py_compile import PycInvalidationMode

py_compile.compile(
    "module.py",
    doraise=True,
    invalidation_mode=PycInvalidationMode.TIMESTAMP,
)

This is fast and is the usual default when SOURCE_DATE_EPOCH is absent. Filesystems with coarse timestamp resolution can create edge cases in which content changes while metadata appears equivalent.

Checked hashes

py_compile.compile(
    "module.py",
    doraise=True,
    invalidation_mode=PycInvalidationMode.CHECKED_HASH,
)

The interpreter hashes the source again during import. This improves determinism and avoids relying only on timestamps, at the cost of additional reading and hashing.

Unchecked hashes

py_compile.compile(
    "module.py",
    doraise=True,
    invalidation_mode=PycInvalidationMode.UNCHECKED_HASH,
)

Python assumes the cache is valid. Use this only when a package manager or build system rigorously updates bytecode whenever source changes.

SOURCE_DATE_EPOCH

When SOURCE_DATE_EPOCH is set, the default invalidation mode becomes CHECKED_HASH. This supports reproducible builds that should not depend on real timestamps.

Since Python 3.7.2, the environment variable determines the default but does not override an explicit argument.

The quiet parameter

quiet controls messages when doraise=False.

• 0 or 1: normal diagnostic behavior;
• 2: suppress messages and make doraise ineffective.

Automation should prefer doraise=True and exception handling rather than complete silence.

Command-line interface

The module can compile explicitly named files.

python -m py_compile file1.py file2.py

It does not recursively search a directory. The exit status is nonzero if any file cannot be compiled.

Read filenames from stdin

When - is the only argument, filenames are read from standard input.

find src -name '*.py' -print | python -m py_compile -

Unix filenames can technically contain newlines. A strict build pipeline may prefer Python’s Path API instead.

Quiet mode in the CLI

python -m py_compile -q file1.py file2.py

The option suppresses diagnostics, but a pipeline must still check the process exit code.

Permissions and shared installations

Precompilation is useful when end users can read a package but cannot write to its __pycache__ directory.

An installer with appropriate privileges creates the cache, after which permissions should permit reading without granting unnecessary write access.

Concurrent writes

The temporary-write-and-rename strategy reduces partially written caches when processes compile the same target. It does not make it safe to direct unrelated source files to one custom cfile.

Use separate build directories per interpreter and avoid concurrent jobs writing the same custom artifact.

pyc files do not protect source logic

Bytecode can be inspected and disassembled. Shipping only .pyc files does not provide strong intellectual-property protection, signing, or encryption.

Untrusted bytecode remains dangerous when imported, just like untrusted source.

Remove stale caches

Most projects should not commit __pycache__. When changing interpreters or creating a clean build, remove caches and regenerate them.

from pathlib import Path

for folder in Path(".").rglob("__pycache__"):
    for file in folder.iterdir():
        file.unlink()
    folder.rmdir()

Use cleanup code carefully in shared environments.

A controlled compiler helper

from pathlib import Path
import py_compile


def compile_file(file: Path) -> Path:
    output = py_compile.compile(
        str(file),
        doraise=True,
        optimize=0,
        invalidation_mode=py_compile.PycInvalidationMode.CHECKED_HASH,
    )
    return Path(output)

The caller can record the artifact size, digest, and Python version.

Testing

Create temporary valid and invalid sources and verify destinations, exceptions, and invalidation policy.

from tempfile import TemporaryDirectory

with TemporaryDirectory() as folder:
    source = Path(folder) / "ok.py"
    source.write_text("value = 42\n", encoding="utf-8")
    pyc = compile_file(source)
    assert pyc.exists()

Also test a symlink destination on supported platforms.

Common mistakes

• Assuming pyc files are portable across Python versions.
• Ignoring a None return when doraise=False.
• Using one custom output for several sources.
• Choosing unchecked hashes without a trusted build system.
• Shipping pyc as source-code protection.
• Confusing compilation with functional testing.
• Failing to record optimization level.
• Committing locally generated caches unnecessarily.

Best practices

• Use doraise=True in automation.
• Prefer standard __pycache__ paths.
• Select invalidation explicitly for reproducible builds.
• Separate artifacts by interpreter and optimization.
• Compile before deployment to catch syntax errors.
• Run tests in addition to compilation.
• Never import untrusted bytecode.
• Clean caches when switching environments.

Conclusion

The Python py_compile module compiles source files into .pyc caches, validating syntax and preparing installations where the runtime cannot write into package directories. It controls destination, traceback filename, optimization, and invalidation.

The choice between timestamps and hashes depends on the build process. With explicit exceptions, conventional paths, and version-isolated artifacts, py_compile makes bytecode generation predictable without confusing a cache with a portable executable, code protection, or functional test.