Python pstats reads, merges, sorts, filters, and formats data produced by cProfile and profile. The profiler records function-call, return, and exception events; pstats turns that raw information into reports that reveal slow call trees, excessive invocation counts, and poor algorithm choices.
Profiling is not benchmarking. A deterministic profiler adds overhead and observes a complete execution. Use timeit for controlled comparisons of small snippets and use pstats to understand where a representative application run spent its time.
Create a profile dump
The cProfile command can save binary statistics to a file:
python -m cProfile -o profile.prof application.pyTo profile a module:
python -m cProfile -o profile.prof -m my_package.cliThe dump is not a stable interchange format. Analyze it with the same Python version, implementation, and preferably operating system that produced it.
Load data with Stats
import pstats
stats = pstats.Stats("profile.prof")
stats.print_stats()The constructor can also receive a cProfile.Profile object, several dump files, or a custom output stream.
Understand the columns
The common report columns are:
ncalls: total function calls.tottime: time spent inside the function, excluding subcalls.percall:tottimedivided by call count.cumtime: time in the function and all functions below it.- cumulative
percall: cumulative time divided by primitive calls.
Recursive functions may show total/primitive call counts.
Sort by cumulative time
SortKey.CUMULATIVE highlights functions whose complete call trees consumed the most time.
from pstats import Stats, SortKey
Stats("profile.prof") \
.sort_stats(SortKey.CUMULATIVE) \
.print_stats(20)This is usually the best first view for identifying expensive endpoints, jobs, or high-level algorithms.
Sort by internal time
SortKey.TIME ranks time spent directly in each function body, excluding callees.
Stats("profile.prof") \
.sort_stats(SortKey.TIME) \
.print_stats(20)High cumulative time with low internal time means the function delegates most cost. High values in both columns mean the function performs substantial work itself.
Use SortKey enums
The enum is more robust than abbreviated strings. Useful keys include CALLS, PCALLS, FILENAME, LINE, NAME, NFL, STDNAME, TIME, and CUMULATIVE.
stats.sort_stats(
SortKey.NAME,
SortKey.FILENAME,
SortKey.LINE,
)Additional keys break ties. Avoid the old numeric sorting interface in new code.
Strip directory prefixes
strip_dirs() shortens reports:
stats.strip_dirs() \
.sort_stats(SortKey.TIME) \
.print_stats()The method modifies the object and may merge entries that become indistinguishable after path removal. Keep an unmodified instance when full paths matter.
Restrict printed results
print_stats() accepts a line count, fraction, and regular-expression filters.
stats.sort_stats(SortKey.CUMULATIVE)
stats.print_stats(30)
stats.print_stats(0.10)
stats.print_stats("my_package")Restrictions are applied in order. Reducing to 50 percent and then filtering by package is different from filtering first and taking half afterward.
Inspect callers
print_callers() shows which functions invoked each selected function.
stats.print_callers("process_order")This view can reveal a cheap helper that became expensive because many code paths call it repeatedly.
Inspect callees
print_callees() shows the opposite direction:
stats.print_callees("process_order")Use callers to locate demand and callees to decompose the work performed by a high-level function.
Merge several runs
The constructor can combine several profile files. Functions with the same file, line, and name are accumulated.
stats = pstats.Stats(
"worker-1.prof",
"worker-2.prof",
"worker-3.prof",
)Add later results with add():
stats.add("worker-4.prof")Merge only comparable runs. Mixing different workloads, Python versions, configurations, or machines can create misleading averages.
Compare before and after
pstats aggregates dumps but does not automatically calculate a statistically meaningful delta between two releases. Generate equivalent reports, export structured records, and compare selected functions.
Repeat each scenario, control input size, record the commit and environment, and account for warm-up and caching effects.
Capture text output
import io
import pstats
from pstats import SortKey
buffer = io.StringIO()
pstats.Stats("profile.prof", stream=buffer) \
.sort_stats(SortKey.CUMULATIVE) \
.print_stats(20)
report = buffer.getvalue()This is convenient for CI artifacts and dashboards. Remove absolute paths and sensitive function names before publishing reports outside the team.
Analyze an in-memory Profile
import cProfile
import pstats
profiler = cProfile.Profile()
profiler.enable()
run_scenario()
profiler.disable()
pstats.Stats(profiler) \
.sort_stats(pstats.SortKey.CUMULATIVE) \
.print_stats(15)This avoids temporary files. Saving a dump is still valuable for long-running jobs and later review.
Export structured data
get_stats_profile() returns a StatsProfile containing FunctionProfile records.
profile = stats.get_stats_profile()
for name, function in profile.func_profiles.items():
print(name, function.cumulative_time)The structured API supports JSON and tables. Function names may collide, so preserve source file and line when building identifiers.
Use the interactive browser
Running pstats as a module opens a line-oriented statistics browser:
python -m pstats profile.profIt can load, sort, filter, and display callers or callees. The interface is built with the same concepts described in the Python cmd guide.
Cumulative time and algorithms
A function with high cumulative time may have selected an expensive algorithm, repeated I/O, or invoked a dependency too often. Micro-optimizing its own lines may not help. Reducing work, calls, allocation, and network round trips often matters more.
Investigate call counts
SortKey.CALLS highlights frequently invoked functions.
stats.sort_stats(SortKey.CALLS).print_stats(20)A tiny helper can dominate runtime when called millions of times. Unexpected counts can also reveal duplicate callbacks or accidental loops.
Profile time and memory separately
pstats analyzes calls and time, not allocations. For memory growth, compare snapshots using Python tracemalloc. Run tools separately when possible to keep overhead and interpretation manageable.
Do not confuse profiles with bytecode
Profile reports are organized by functions, files, and source lines. For compiled instruction analysis, read Python opcode and the Python dis guide.
Import-time performance
When a profile shows heavy import costs, the Python modulefinder guide can map static dependencies. Dynamic imports and top-level module side effects still require runtime measurement.
Deterministic-profiler limitations
- The profiler adds overhead.
- C-level functions may appear disproportionately fast.
- Very short calls accumulate timing error.
- One run may not represent production traffic.
- External I/O varies with the environment.
- Threads and processes need planned collection.
Treat a profile as evidence for hypotheses and confirm improvements with repeated measurements.
Protect profile dumps
Dumps can reveal absolute paths, module names, application architecture, business functions, and usage patterns. Store them with access controls and do not load unknown dumps in sensitive environments.
The format has no guaranteed future compatibility and should not become a public API.
Recommended practices
- Start with
CUMULATIVE, then inspectTIME. - Filter to application code.
- Review callers and callees.
- Repeat representative workloads.
- Record the environment and commit.
- Do not treat profiling as precise benchmarking.
- Protect dump files.
- Measure again after every optimization.
Conclusion
Python pstats transforms profiler output into actionable reports. Sorting, filtering, aggregation, callers, callees, and structured profiles help distinguish directly slow functions from cost caused by delegation or excessive frequency.
Consult the official pstats documentation and the Python timeit documentation.
