The resource module measures and limits resources used by Unix processes. It can report CPU time, page faults, context switches, and peak memory while enforcing quotas for file descriptors, file size, address space, process creation, and other platform-supported resources.
These controls are useful when running plugins, compilers, converters, user jobs, or subprocesses that must not consume CPU, memory, or disk without bounds. They do not replace containers, cgroups, sandboxes, or a dedicated unprivileged account, but they add an important defensive layer.
Availability
resource is available on Unix and not on WASI. Constants vary by platform, so check them with hasattr().
import resource
if hasattr(resource, "RLIMIT_AS"):
print(resource.getrlimit(resource.RLIMIT_AS))
Do not assume Linux, macOS, and BSD expose or enforce the same limits.
Soft and hard limits
Every resource has a (soft, hard) pair. The soft limit is currently enforced. A process can normally lower it and raise it again up to the hard limit. Raising the hard limit requires appropriate privileges.
soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
print("Soft:", soft, "Hard:", hard)
RLIM_INFINITY represents unlimited when supported by the system.
Avoid lowering the hard limit unnecessarily
After an unprivileged process lowers a hard limit, it cannot restore it. For temporary self-limiting, keep the hard value and change only the soft limit.
soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
resource.setrlimit(resource.RLIMIT_NOFILE, (min(256, hard), hard))
Even changing the main process’s soft limit may break libraries loaded later. Prefer applying limits inside a dedicated child process.
Limit file descriptors
RLIMIT_NOFILE controls how many descriptors can be open.
def limit_fds(maximum=128):
soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
value = min(maximum, hard)
resource.setrlimit(resource.RLIMIT_NOFILE, (value, hard))
When exhausted, operations may raise OSError with EMFILE. Close files, sockets, and pipes correctly; a quota is not a cleanup strategy.
Limit CPU time
RLIMIT_CPU counts CPU seconds rather than wall-clock time.
resource.setrlimit(resource.RLIMIT_CPU, (2, 3))
Exceeding the soft value sends SIGXCPU. Continuing until the hard value may terminate the process.
A sleeping or network-bound task can run much longer than two seconds while consuming little CPU. Use an external wall-clock timeout as well.
Handle SIGXCPU
import signal
def cpu_exceeded(signum, frame):
save_minimal_state()
raise SystemExit(124)
if hasattr(signal, "SIGXCPU"):
signal.signal(signal.SIGXCPU, cpu_exceeded)
Keep handlers short. Python signal explains why locks and complex logging are unsafe there.
Limit file size
RLIMIT_FSIZE limits the maximum size of a file created by the process.
limit = 10 * 1024 * 1024
resource.setrlimit(resource.RLIMIT_FSIZE, (limit, limit))
This reduces the risk of one huge output file but does not limit file count or all filesystem usage. Use a quota or isolated directory for stronger protection.
Limit address space
RLIMIT_AS limits virtual address space.
if hasattr(resource, "RLIMIT_AS"):
limit = 512 * 1024 * 1024
resource.setrlimit(resource.RLIMIT_AS, (limit, limit))
Scientific libraries, mmap, shared objects, and allocators reserve virtual space that is not equal to resident memory. A low value may prevent imports or raise MemoryError before useful work begins.
RLIMIT_DATA and RLIMIT_RSS
RLIMIT_DATA traditionally limits the heap, but modern allocations may bypass it. RLIMIT_RSS may be advisory or weakly enforced.
For production memory control, container or cgroup limits are usually more predictable.
Stack size
RLIMIT_STACK limits the main thread’s stack. Lowering it too far can crash recursive code, C extensions, or library initialization.
It is not a safe substitute for controlling Python recursion. sys.setrecursionlimit() addresses a different layer.
Process count
RLIMIT_NPROC limits process creation according to platform rules.
if hasattr(resource, "RLIMIT_NPROC"):
soft, hard = resource.getrlimit(resource.RLIMIT_NPROC)
resource.setrlimit(resource.RLIMIT_NPROC, (min(32, hard), hard))
Threads may count on some systems, and services sharing a user account may compete for the same quota.
Apply limits in a child wrapper
preexec_fn exists on Unix but can be unsafe in a multithreaded parent. A simple wrapper executable is easier to reason about.
# limited_wrapper.py
import os
import resource
import sys
resource.setrlimit(resource.RLIMIT_CPU, (2, 3))
resource.setrlimit(resource.RLIMIT_FSIZE, (10_000_000, 10_000_000))
os.execvp(sys.argv[1], sys.argv[1:])
The parent starts the wrapper with an argument list and no shell.
prlimit()
On Linux, prlimit() reads or changes another process’s limits when permitted.
if hasattr(resource, "prlimit"):
previous = resource.prlimit(pid, resource.RLIMIT_NOFILE, (128, 128))
print(previous)
It can raise ProcessLookupError when the PID disappears and PermissionError without sufficient capability. Verify process identity because PIDs are reused.
Auditing
setrlimit() and prlimit() raise auditing events. Restricted runtimes may log or block changes.
getrusage()
getrusage() reports the current process, reaped children, or current thread when supported.
usage = resource.getrusage(resource.RUSAGE_SELF)
print("User CPU:", usage.ru_utime)
print("System CPU:", usage.ru_stime)
print("Peak RSS:", usage.ru_maxrss)
RUSAGE_SELF includes all threads. RUSAGE_CHILDREN includes terminated and waited-for children. RUSAGE_THREAD may report the current thread.
ru_maxrss units
The unit of ru_maxrss is platform-dependent. Linux normally reports KiB, while macOS reports bytes. Normalize before showing MB or comparing hosts.
import platform
value = usage.ru_maxrss
rss_bytes = value if platform.system() == "Darwin" else value * 1024
Validate this rule on other Unix systems.
Peak is not current memory
ru_maxrss is the highest observed resident usage. It never decreases when memory is released.
Use procfs, psutil, container metrics, or a system monitor for a time series of current memory.
CPU accounting
ru_utime is user-mode CPU and ru_stime is kernel-mode CPU.
before = resource.getrusage(resource.RUSAGE_SELF)
run_work()
after = resource.getrusage(resource.RUSAGE_SELF)
cpu = (after.ru_utime - before.ru_utime) + (after.ru_stime - before.ru_stime)
This measures accumulated CPU, not a controlled benchmark. Use timeit for microbenchmarks.
Page faults
ru_minflt counts faults not requiring I/O, while ru_majflt counts faults requiring storage access.
Results depend on cache, kernel, and environment. One run is not enough to prove a regression.
Context switches
ru_nvcsw counts voluntary switches and ru_nivcsw involuntary switches. Many involuntary switches can indicate CPU contention; many voluntary switches often reflect waiting or synchronization.
Block operations
ru_inblock and ru_oublock represent block I/O operations when supported. They are not byte counts.
Children must be reaped
For usage to appear under RUSAGE_CHILDREN, a child must terminate and be collected with wait() or waitpid().
Page size
getpagesize() returns the system page size used by this interface.
print(resource.getpagesize())
It does not necessarily describe huge pages or every hardware detail.
Limits are not a sandbox
A limited process may still read allowed files, access the network, inherit environment variables, and use credentials. Combine rlimits with an unprivileged user, isolated directories, executable allowlists, and OS sandboxing or containers.
Apply limits early
Set limits before loading or executing untrusted content. Lowering a quota does not undo files already opened or memory already allocated.
Diagnostics
Log effective limits at child startup. When a process exits by signal, report that CPU, file-size, or memory policy may be involved without claiming certainty. SIGKILL can also come from an operator.
Testing
Test missing constants, soft greater than hard, attempts to raise hard limits, CPU-bound and sleeping tasks, file growth, descriptor exhaustion, library imports under RLIMIT_AS, child usage, threads, Linux/macOS, and containers.
Common mistakes
Common failures include limiting the main application, confusing CPU time with wall time, treating ru_maxrss as current memory, ignoring platform units, lowering hard limits permanently, assuming RLIMIT_RSS is strict, and calling rlimits a complete sandbox.
Conclusion
resource measures consumption and enforces basic Unix quotas. It works best in dedicated child processes with limits set before work and external monitoring for wall time and memory.
Check every constant on the target system, normalize metrics, and combine rlimits with real isolation. Consult the official resource documentation and getrlimit(2).
