Python tracemalloc helps you discover where a program allocates memory. It records allocation traces and lets you compare snapshots to identify unexpected growth, retained objects, and code paths that deserve investigation. It is particularly useful in APIs, workers, data pipelines, command-line services, and applications that remain active for hours or days.
Growing memory does not automatically mean a leak. A process may intentionally keep caches, connection pools, buffers, imported modules, or global indexes. The purpose of tracemalloc is therefore not merely to prove that memory increased, but to show which files, lines, and call stacks contributed to that increase.
How tracemalloc works
After tracing starts, Python records the origin of memory blocks handled by its allocator. A record may contain a filename, line number, and a short traceback. Your program can then capture snapshots and group the data by line, file, or traceback.
import tracemalloc
tracemalloc.start()
values = [str(number) for number in range(100_000)]
snapshot = tracemalloc.take_snapshot()
for stat in snapshot.statistics('lineno')[:10]:
print(stat)The lineno grouping highlights lines responsible for the largest tracked allocations.
Start tracing early
Enable tracing near process startup when you need to include imports, framework initialization, and cache construction. You may call tracemalloc.start() or set the PYTHONTRACEMALLOC environment variable.
PYTHONTRACEMALLOC=10 python app.pyThe number controls the maximum number of frames stored for each trace. Deeper traces provide more context but consume additional memory and CPU.
Compare snapshots
The most practical workflow is to capture a baseline, execute the suspected operation repeatedly, and capture a second snapshot.
tracemalloc.start(10)
before = tracemalloc.take_snapshot()
for _ in range(50):
process_batch()
after = tracemalloc.take_snapshot()
for diff in after.compare_to(before, 'lineno')[:20]:
print(diff)The comparison reports size differences, block counts, and source locations. Consistent growth at the same location is a strong investigation signal.
Build a reproducible workload
Memory diagnostics are more trustworthy when the workload is controlled. Run the same operation multiple times with similar input and avoid mixing deployment activity, imports, or one-time warm-up behavior with the main measurement.
For an API, warm routes, database pools, and serializers before the baseline. For a worker, process a few initial jobs so queues and internal caches stabilize.
Filter irrelevant traces
Snapshots include application code, libraries, test frameworks, and runtime infrastructure. Apply filters to focus on your project.
filters = [
tracemalloc.Filter(True, '*/my_project/*'),
tracemalloc.Filter(False, '*/site-packages/*'),
]
filtered = snapshot.filter_traces(filters)Verify paths in the real environment because containers and virtual environments change directory prefixes.
Group by traceback
Grouping by traceback shows the call sequence that produced an allocation. This matters when the same helper is called from multiple routes or jobs.
for stat in snapshot.statistics('traceback')[:5]:
print(stat)
for line in stat.traceback.format():
print(line)Begin with a shallow trace and increase depth only when the final line lacks enough context.
Read current and peak usage
get_traced_memory() returns current tracked memory and the highest value observed since tracing began.
current, peak = tracemalloc.get_traced_memory()
print(current / 1024 / 1024)
print(peak / 1024 / 1024)Peak usage is valuable for operations that release memory afterward but require a large temporary buffer.
Reset the peak for one operation
tracemalloc.reset_peak()
result = build_report()
current, peak = tracemalloc.get_traced_memory()This resets only the peak reference. It does not clear allocations or restart tracing.
Unbounded global collections
A common leak-like pattern is a global list that grows for the lifetime of the process.
history = []
def record(event):
history.append(event)Tracemalloc will point to the append line. A bounded deque, external storage, or expiration policy may solve the problem.
from collections import deque
history = deque(maxlen=10_000)Bound your caches
An unlimited cache can retain arguments and results indefinitely. Prefer explicit limits and observe hit rates.
from functools import lru_cache
@lru_cache(maxsize=512)
def load_configuration(key):
...Inspect cache_info() and clear caches when their lifecycle requires it.
Callbacks can retain objects
Closures, listeners, and callbacks may keep large structures alive. A callback registered in a global registry can capture an object unexpectedly. Review signal handlers, task registries, event buses, and pending futures.
The guide to Python weakref explains how weak references help with caches and observer registries.
Generators and asynchronous tasks
Suspended generators retain their frames and local variables. Pending asyncio tasks can retain context, exceptions, request bodies, and response buffers. List pending tasks, inspect queues, and ensure canceled tasks are awaited.
For isolated asynchronous context, see Python contextvars.
Exceptions and tracebacks
Keeping exception objects for long periods can preserve frames and local variables. Avoid placing complete exceptions in global lists. Serialize the information needed for logs and metrics instead.
The article about Python traceback covers safe stack formatting without unnecessary retention.
Large reads and temporary buffers
Calling read() without a size may load an entire file. Prefer streaming or chunked processing for large data. See Python tempfile for secure temporary-file patterns.
Tracemalloc does not measure everything
The module mainly tracks allocations made through Python’s memory manager. Native memory owned by C extensions, numerical libraries, drivers, graphics stacks, or child processes may not appear completely.
Compare tracemalloc with resident set size, operating-system metrics, and library-specific profilers. In NumPy-heavy workloads, native buffers may dominate total process memory.
Use complementary tools
Combine allocation snapshots with RSS monitoring, object inspection, logging, and load tests. The official tracemalloc documentation explains snapshots, filters, and traces. The CPython memory management documentation describes internal allocators.
Save snapshots
snapshot.dump('/tmp/memory.snapshot')
loaded = tracemalloc.Snapshot.load('/tmp/memory.snapshot')Saved snapshots support later comparison, but treat them as internal diagnostic data because they may expose file paths and project structure.
Create regression tests
A test can repeat a workload and check whether growth remains within a reasonable margin.
def test_memory_growth_is_bounded():
tracemalloc.start()
before = tracemalloc.take_snapshot()
for _ in range(100):
run_flow()
after = tracemalloc.take_snapshot()
growth = sum(
item.size_diff
for item in after.compare_to(before, 'filename')
)
assert growth < 5 * 1024 * 1024Avoid overly strict thresholds because Python and dependency versions may change allocation patterns. Treat the test as a regression signal rather than absolute proof.
Production precautions
Tracing adds overhead. Enable it for a limited window, on a diagnostic replica, or with a shallow traceback. Do not accumulate snapshots indefinitely. Remove temporary files and call tracemalloc.stop() after the investigation.
Investigation checklist
- Reproduce the workload under controlled conditions.
- Warm the application before the baseline.
- Compare snapshots with the same grouping.
- Filter unrelated dependencies.
- Inspect caches, global collections, and registries.
- Review tasks, generators, and retained exceptions.
- Compare tracked memory with RSS and native memory.
- Validate the fix with another run.
Conclusion
Python tracemalloc turns vague leak suspicions into actionable evidence. Snapshot comparisons, filters, and allocation tracebacks reveal which lines keep growing and help distinguish legitimate caches from accidental retention.
Use it together with system metrics and reproducible tests. This combination produces faster diagnoses, safer fixes, and long-running Python services with predictable memory behavior.
