The codeop module helps compile Python source received interactively. Its key feature is distinguishing three states: complete valid code, definitely invalid code, and code that is incomplete but may become valid after the user enters more lines. That decision is essential for REPLs, embedded consoles, notebooks, educational editors, administrative shells, and tools that execute multiline commands.
A plain compile() call is not designed to decide whether a block merely needs additional input. codeop packages heuristics compatible with Python’s interactive behavior and can also remember from __future__ effects between commands through CommandCompiler.
The incomplete-code problem
Consider a function entered one line at a time:
def double(value):
return value * 2
After the first line, the console should not reject the input. It should display a continuation prompt. After the body and the appropriate terminator, the block can be compiled and executed.
compile_command
compile_command(source, filename="<input>", symbol="single") determines the current state.
import codeop
result = codeop.compile_command("1 + 2")
print(result)
Complete code produces a code object. Apparently incomplete code returns None. Definitely invalid code raises a syntax-related exception.
Handle three outcomes
def analyze(source):
try:
code = codeop.compile_command(source)
except (SyntaxError, OverflowError, ValueError) as error:
return "invalid", error
if code is None:
return "incomplete", None
return "complete", code
Do not treat None as a failure. It tells the interface to continue collecting lines.
The symbol parameter
symbol controls the expected input form. Common values are single, exec, and eval.
singlerepresents interactive input and can trigger expression display behavior.execrepresents a suite of statements.evalaccepts one expression.
expression = codeop.compile_command(
"10 * 4",
filename="<calculator>",
symbol="eval",
)
Choose the mode deliberately
A traditional console generally uses single. A cell-based editor may use exec. A calculator may use eval, although evaluating untrusted expressions remains dangerous.
The mode defines grammar and code-object behavior. It does not provide security.
Accumulate lines
A minimal REPL keeps a buffer until input becomes complete.
import codeop
lines = []
while True:
prompt = "... " if lines else ">>> "
line = input(prompt)
lines.append(line)
source = "\n".join(lines)
try:
code = codeop.compile_command(source)
except SyntaxError as error:
print(f"error: {error}")
lines.clear()
continue
if code is None:
continue
exec(code, namespace)
lines.clear()
A real console also needs EOF, KeyboardInterrupt, history, encoding, output, and execution-exception policies.
Blank lines
Interactive consoles use blank lines to finish selected compound statements. Preserve that behavior rather than deleting all empty input.
Do not call strip() on the full buffer because it can alter indentation and termination.
Indentation
Leading whitespace is Python syntax. Preserve exactly what the user entered.
An editor may suggest indentation, but silently rewriting source before compilation can create surprising behavior.
SyntaxError
Definitely invalid code raises SyntaxError.
try:
codeop.compile_command("if :")
except SyntaxError as error:
print(error.msg, error.lineno, error.offset)
Display filename, line, column, and a short excerpt without exposing another user’s buffer.
Other compilation failures
Extreme input can raise OverflowError, ValueError, or resource-related failures. Limit bytes, lines, nesting, and compilation time before accepting public input.
Do not raise recursion limits automatically in response to hostile or malformed source.
Symbolic filenames
The filename argument appears in tracebacks and diagnostics.
code = codeop.compile_command(
source,
filename="<admin-console>",
)
Use a name that identifies the session or cell without exposing personal data. A stable cell ID helps retrieve matching source later.
Integration with linecache
Dynamically generated code has no ordinary file. To show correct traceback lines, preserve source under its symbolic filename and integrate carefully with source caching.
See Python linecache.
CommandCompiler
CommandCompiler is a stateful alternative to calling compile_command() directly.
import codeop
compiler = codeop.CommandCompiler()
code = compiler("x = 10")
It is designed for successive commands belonging to one interactive session.
Future statements
Inside a module, from __future__ import ... affects later code in the file. In a console, it should affect later commands in the same session.
CommandCompiler remembers future flags observed during earlier compilations.
compiler = codeop.CommandCompiler()
compiler("from __future__ import annotations")
next_code = compiler("def f(x: Type) -> Other: pass")
One compiler per session
Do not share one CommandCompiler across independent users. Future flags and session behavior could leak between contexts.
Create one compiler for each console, notebook kernel, connection, or tenant.
Compilation versus execution
Compilation validates syntax and creates bytecode; it does not execute the body. Execution begins with exec() or eval().
Compilation itself can consume resources, but execution introduces the larger risks: filesystem, network, imports, subprocesses, introspection, and native code.
It is not a sandbox
codeop does not restrict Python. Using eval mode, reducing builtins, or rejecting a few AST nodes does not create a secure language.
For untrusted code, use strong isolation: a separate process, unprivileged account, restricted filesystem, controlled network, CPU and memory limits, and a hard deadline.
Persistent namespaces
A console usually keeps one globals dictionary between commands.
namespace = {"__name__": "__console__"}
exec(code, namespace, namespace)
This allows variables and functions to persist, but it also allows memory, files, and references to accumulate.
Separate users
Every user needs an independent namespace. Shared globals allow one session to read or modify another session’s data.
Destroy the worker process when a session ends if strong cleanup matters. Threads and native resources may survive simple dictionary deletion.
Interactive expression display
single mode cooperates with the interpreter’s display mechanism. A custom console can adjust sys.displayhook to control formatting and history.
Use reprlib to bound huge or recursive values. See Python reprlib.
Large results
An expression can produce an enormous or recursive representation. Limit output bytes, lines, and rendering time.
A hostile object’s __repr__ can execute arbitrary logic. Do not render untrusted objects in a privileged host process.
Capturing stdout and stderr
Web consoles often capture output. contextlib.redirect_stdout() changes process-global state and is unsafe for several concurrent sessions in one interpreter.
One worker process per session provides clearer ownership of streams, signals, limits, and termination.
Execution exceptions
After compilation, catch execution errors at the correct boundary.
try:
exec(code, namespace, namespace)
except SystemExit:
close_session()
except Exception:
traceback.print_exc()
Define explicit behavior for KeyboardInterrupt, SystemExit, and cancellation. Avoid a blanket handler that prevents application shutdown.
Execution timeouts
A thread cannot safely stop arbitrary Python and native code. Use disposable processes and terminate the worker after a deadline.
For trusted internal sessions, a faulthandler dump before termination can preserve diagnostic context.
Async and top-level await
An asynchronous console may want top-level await. That requires compilation flags and event-loop support beyond basic codeop behavior.
Use APIs provided by the target Python version or interactive framework rather than inventing partial coroutine execution.
Notebooks
Notebook kernels manage sessions, history, display protocols, cell IDs, asynchronous execution, source storage, and rich outputs. codeop solves only syntactic completeness and future flags.
A simple input() loop is not a complete notebook kernel.
Administrative consoles
An embedded console is a high-risk management surface. Require strong authentication, authorization, audit logs, a private network, and temporary access.
Prefer explicit administrative commands over arbitrary Python execution whenever possible.
History
If commands are stored, encrypt them and define retention. Users may enter tokens, personal data, and secrets accidentally.
Provide deletion controls and avoid logging sensitive sessions by default.
Input limits
Limit bytes, lines, nesting, and how long a client may keep an incomplete buffer. Otherwise, a client can occupy memory and a connection indefinitely.
After a deadline, discard the buffer or close the session.
Cancel the current buffer
Offer a command that clears only the unfinished block.
if line == ":cancel":
lines.clear()
continue
Choose commands outside normal Python syntax and document them.
Autocomplete
Completeness detection does not provide autocomplete. Suggestions require tokens, partial ASTs, symbol tables, or a language server.
Avoid evaluating descriptors and properties merely to discover attributes because that can trigger side effects.
Version compatibility
Python grammar and incomplete-input heuristics evolve. Test the console on each supported version.
Use the codeop implementation shipped with the interpreter rather than copying old internal logic.
Testing
Cover simple expressions, multiline functions and classes, decorators, open parentheses, triple strings, comprehensions, try/except, pattern matching, async syntax, definitive errors, EOF, interruption, and future imports.
Also verify that independent sessions do not share namespaces or compiler flags.
Common mistakes
Common failures include treating None as an error, stripping indentation, choosing the wrong symbol, sharing CommandCompiler, executing hostile code in the host process, capturing global streams, allowing unlimited buffers, and mistaking compilation for sandboxing.
Conclusion
codeop provides the missing logic for deciding whether interactive Python input is complete, incomplete, or invalid. Use compile_command() for stateless checks and CommandCompiler for sessions that preserve __future__ flags.
Separate namespaces, retain source for tracebacks, enforce limits, and isolate execution. Consult the official codeop documentation and Python symtable for analyzing names in a session.
