Temporary files appear in exports, uploads, tests, conversions, downloads, image processing, and communication with external programs. Creating them manually in a fixed directory may look easy, but it can cause name collisions, weak permissions, leftover data, and race-condition vulnerabilities. The Python tempfile module provides secure, portable APIs for temporary files and directories with automatic cleanup.
This guide covers TemporaryFile, NamedTemporaryFile, SpooledTemporaryFile, TemporaryDirectory, mkstemp(), and mkdtemp(), including Windows and Unix differences, binary and text modes, visible names, cleanup, testing, and security. It complements our articles about Python lists, slicing, collections, performance, and memory management.
Why manual temporary names are unsafe
Code such as open('/tmp/output.txt', 'w') may work in an isolated test, but it is fragile in production. Two processes can choose the same name, another process can create a file or symbolic link before your program, and a hard-coded directory may not exist or be writable on another platform.
tempfile combines secure name selection and creation in one operation. Generated names contain random characters, and permissions follow rules intended for the creating user.
TemporaryFile for simple storage
TemporaryFile() returns a file-like object. Its default mode is binary read/write, w+b, and the resource is removed when closed.
from tempfile import TemporaryFile
with TemporaryFile() as file:
file.write(b"temporary result")
file.seek(0)
data = file.read()
print(data)The with block guarantees closing and cleanup even when an exception occurs. On Unix, the file may never have a visible directory entry. Other platforms may use a named file internally. Do not make application logic depend on whether a path is visible.
Text mode and encoding
For textual data, declare the mode and encoding explicitly.
from tempfile import TemporaryFile
with TemporaryFile(mode="w+t", encoding="utf-8") as file:
file.write("Hello, temporary file!\n")
file.seek(0)
print(file.read())Explicit parameters prevent encoding differences between machines. Keep binary mode for images, PDFs, archives, and network payloads.
NamedTemporaryFile when a path is required
Some libraries and command-line tools accept only a filename, not an open stream. NamedTemporaryFile() creates a visible path and exposes it through name.
from tempfile import NamedTemporaryFile
with NamedTemporaryFile(suffix=".json") as file:
print(file.name)
file.write(b'{"status": "ok"}')
file.flush()
# pass file.name to another APIA suffix is useful when an external tool detects the format from the extension. You can also set prefix and dir; keyword arguments make these calls easier to read.
delete and delete_on_close
By default, a named temporary file is deleted when it is closed. Since Python 3.12, delete_on_close separates object closing from cleanup at context-manager exit.
from tempfile import NamedTemporaryFile
with NamedTemporaryFile(
mode="w+b",
suffix=".bin",
delete=True,
delete_on_close=False,
) as file:
path = file.name
file.write(b"data")
file.close()
with open(path, "rb") as reader:
print(reader.read())
# removed when the outer context exitsThis pattern helps when another library must reopen the file by name. Setting delete=False transfers full cleanup responsibility to the application.
Important Windows differences
POSIX systems generally allow a file to be reopened or unlinked while it remains open. Windows applies stricter sharing and delete-access rules. Reopening a still-open NamedTemporaryFile can fail depending on delete, delete_on_close, and how the second handle is created.
A predictable strategy is to use delete_on_close=False, close the original object before reopening by name, and close every additional handle before leaving the context. Test the behavior on the same operating system used in production.
TemporaryDirectory for a complete workspace
When a job uses several files, create an isolated temporary directory.
from pathlib import Path
from tempfile import TemporaryDirectory
with TemporaryDirectory(prefix="report-") as folder:
root = Path(folder)
source = root / "input.csv"
result = root / "output.json"
source.write_text("name,value\nA,10\n", encoding="utf-8")
result.write_text('{"total": 10}', encoding="utf-8")
print(list(root.iterdir()))
# directory and contents removedThe context value contains the path, and cleanup is recursive. This is ideal for import tests, controlled extraction, media conversion, and build pipelines.
delete and ignore_cleanup_errors
TemporaryDirectory accepts ignore_cleanup_errors for best-effort cleanup, which can help when Windows still has an open file. Since Python 3.12, delete=False can preserve the tree after context exit for debugging.
from tempfile import TemporaryDirectory
with TemporaryDirectory(delete=False) as folder:
print("Preserved for inspection:", folder)Do not keep this behavior in production without a retention policy. Abandoned temporary trees can fill the disk.
SpooledTemporaryFile: memory before disk
SpooledTemporaryFile keeps data in memory until it exceeds max_size or an operation such as fileno() requires a real file. It then rolls over to disk.
from tempfile import SpooledTemporaryFile
with SpooledTemporaryFile(max_size=1024 * 1024) as file:
file.write(b"small content")
file.seek(0)
print(file.read())This class works well for uploads, generated responses, and transformations that are usually small but can grow. It combines fast memory access with a safe disk fallback.
Forcing rollover
Call rollover() when another API requires a real file descriptor.
with SpooledTemporaryFile(max_size=10_000) as file:
file.write(b"abc")
file.rollover()
print(file.fileno())Do not depend on the internal _file attribute. Use the public file interface.
mkstemp(): the low-level file API
mkstemp() creates a file securely and returns an operating-system descriptor plus an absolute path.
import os
from tempfile import mkstemp
fd, path = mkstemp(suffix=".txt", prefix="job-")
try:
with os.fdopen(fd, "w", encoding="utf-8") as file:
file.write("content")
finally:
if os.path.exists(path):
os.unlink(path)The function does not remove the file automatically. Closing the descriptor does not delete the pathname. Prefer high-level context managers unless you need descriptor-level control.
mkdtemp(): manual directory cleanup
mkdtemp() securely creates a directory and returns an absolute path. Your code must remove the tree.
import shutil
from pathlib import Path
from tempfile import mkdtemp
folder = Path(mkdtemp(prefix="job-"))
try:
(folder / "result.txt").write_text("ok", encoding="utf-8")
finally:
shutil.rmtree(folder, ignore_errors=False)When automatic cleanup is acceptable, TemporaryDirectory is simpler and less likely to leak resources.
Never use mktemp()
The official tempfile documentation marks mktemp() as deprecated and unsafe. It produces a name that did not exist at the moment of the call but does not create the file immediately. Another process can claim the name before your application opens it.
Use NamedTemporaryFile or mkstemp(), which choose the name and create the resource atomically.
Choosing prefix, suffix, and dir
prefix helps identify resources during debugging, suffix preserves an extension required by external tools, and dir controls placement.
with NamedTemporaryFile(
prefix="thumbnail-",
suffix=".png",
dir="/controlled/path",
) as image:
passThe directory must exist and be writable. Do not build prefixes from untrusted input. Even when the random part is safe, malicious text can confuse logs or surrounding tools.
Finding the default temporary directory
gettempdir() returns the selected default directory. Python considers environment variables such as TMPDIR, TEMP, and TMP, followed by platform-specific locations.
from tempfile import gettempdir
print(gettempdir())Do not assume it is always /tmp. Containers, services, and enterprise systems may redirect temporary storage to another volume. Avoid globally changing tempfile.tempdir; pass dir to the specific operation instead.
flush, seek, and visibility
After writing, call seek(0) before reading from the same stream. If another process or library opens the same path, call flush() first so Python’s buffers reach the operating system.
with NamedTemporaryFile() as file:
file.write(b"123")
file.flush()
file.seek(0)
assert file.read() == b"123"os.fsync() can request stronger persistence, but that is rarely useful for temporary data and has additional cost.
Temporary storage for uploads
Do not keep unlimited uploads entirely in memory. A practical design streams the request into SpooledTemporaryFile, enforces a size limit, validates the actual format, processes it, and cleans up at the end.
Temporary does not mean trusted. Continue applying limits, malware scanning when appropriate, content validation, and protections against decompression bombs or malformed files.
Using temporary files with subprocesses
When a program accepts standard input, piping avoids a path entirely. If it requires a filename, use NamedTemporaryFile or TemporaryDirectory, flush and close according to platform rules, pass the path as a separate argument, and never build a shell command by string concatenation.
When cleanup is manual, place it in finally. A failed external process should not leave sensitive data on disk.
Testing with TemporaryDirectory
Temporary directories make tests independent from the developer’s machine.
from pathlib import Path
from tempfile import TemporaryDirectory
def save(path, text):
Path(path).write_text(text, encoding="utf-8")
with TemporaryDirectory() as folder:
destination = Path(folder) / "test.txt"
save(destination, "value")
assert destination.read_text(encoding="utf-8") == "value"Each test receives an isolated workspace, reducing collisions and repository pollution. Test frameworks may offer fixtures for the same purpose, but understanding tempfile remains useful in scripts and dependency-free tests.
Security and sensitive data
The module creates names securely, but the application must still control directory permissions, retention time, and logging. Do not log paths containing sensitive identifiers. Avoid shared volumes without isolation, and consider encryption when the threat model requires it.
On POSIX, abrupt termination with SIGKILL can prevent cleanup of named files. Long-running services should monitor disk use and remove legitimate stale resources according to age and ownership.
Common mistakes
- Using fixed names in shared directories.
- Choosing
mktemp()because it returns only a path. - Forgetting
seek(0)before reading. - Skipping
flush()before another process opens the file. - Assuming POSIX behavior on Windows.
- Using
delete=Falsewithout later cleanup. - Keeping unlimited uploads in memory.
- Assuming the temporary directory is always
/tmp.
Best practices
- Prefer context managers and high-level APIs.
- Use
TemporaryFilewhen no path is required. - Use
NamedTemporaryFilewhen another API needs a name. - Use
TemporaryDirectoryfor multiple artifacts. - Use
SpooledTemporaryFilefor bounded small content. - Test reopening behavior on Windows.
- Enforce size limits and retention policies.
- Review the official pathlib documentation.
Conclusion
The Python tempfile module removes much of the dangerous work involved in temporary storage. It generates collision-resistant names, creates resources securely, offers automatic cleanup, and behaves consistently across supported platforms.
The right class depends on the integration: TemporaryFile for a disposable stream, NamedTemporaryFile for tools that require a path, SpooledTemporaryFile for balancing memory and disk, and TemporaryDirectory for multi-file pipelines. With context managers, limits, cross-platform tests, and predictable cleanup, temporary storage stops being a source of leaks, permission failures, and security bugs.
