The posix module exposes common system calls on POSIX-compatible platforms. It provides a low-level interface for files, directories, processes, descriptors, permissions, users, groups, and operating-system information. Most application code should import os instead, because that module offers a more portable layer and includes nearly all of the useful posix interface.
Understanding posix still helps explain how Python communicates with Unix systems such as Linux, macOS, and BSD. It is useful when investigating platform-specific behavior, building administration tools, comparing interpreter builds, or learning where many os functions originate.
Availability and portability
posix exists only when Python is built for an operating system that provides the corresponding interface. It is normally unavailable on Windows. A cross-platform library should therefore avoid importing it globally without a fallback.
try:
import posix
except ImportError:
posix = None
if posix is None:
print("POSIX interface is unavailable")
Even on Unix, individual functions can vary with the kernel, C library, compile options, and Python version. Check features with hasattr() or, preferably, use the capability sets published by os.
Why os is usually better
The os module selects the correct implementation for the current platform. On Unix, many os functions are backed by posix; on Windows, another implementation provides equivalent operations. Calls such as os.open(), os.stat(), and os.getpid() therefore keep a consistent public API.
import os
print(os.getpid())
print(os.getcwd())
Import posix directly only when the purpose is explicitly to study, test, or depend on POSIX behavior. For ordinary programs, os is clearer and safer.
File descriptors
Low-level calls operate on integer file descriptors. open() opens a path and returns a descriptor; read() and write() transfer bytes; close() releases the resource.
import os
fd = os.open("data.bin", os.O_RDONLY)
try:
block = os.read(fd, 4096)
print(len(block))
finally:
os.close(fd)
The finally block matters whenever descriptors are managed manually. For regular files, the language-level open() function with with is more convenient because it handles closing, buffering, text decoding, and encoding.
Open flags
POSIX calls combine flags such as O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_EXCL, O_APPEND, and, where available, O_CLOEXEC. Use bitwise OR to build the requested configuration.
import os
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL
fd = os.open("new.txt", flags, 0o600)
try:
os.write(fd, b"private content\n")
finally:
os.close(fd)
O_EXCL together with O_CREAT avoids replacing an existing file and reduces race conditions. Mode 0o600 requests owner-only access, although the process umask also affects final permissions.
Paths as str or bytes
On Unix, paths may be passed as str or bytes. Use text paths for most programs. Bytes are useful only when preserving names that cannot be decoded with the filesystem encoding.
Do not mix the two forms in one operation. Functions given a bytes path normally return bytes names. os.fsencode() and os.fsdecode() convert through Python’s filesystem policy.
File information with stat
stat() returns size, timestamps, owner, group, mode, and additional fields. Use the stat module to interpret mode bits.
import os
import stat
info = os.stat("data.bin", follow_symlinks=False)
print(info.st_size)
print(stat.filemode(info.st_mode))
print(stat.S_ISREG(info.st_mode))
When security depends on the object being inspected, avoid the sequence “check a path, then open it.” Another process could replace the path between those calls. Prefer descriptor-based APIs and parameters such as dir_fd and follow_symlinks when supported.
Symbolic links
stat() normally follows symbolic links, while lstat() describes the link itself. The distinction matters in scanners, backup programs, installers, and cleanup tools.
import os
target = os.readlink("shortcut")
link_info = os.lstat("shortcut")
print(target, link_info.st_mode)
Never assume that a trusted pathname continues pointing to the same object. In shared directories, symlink-swap attacks can redirect privileged operations.
Permissions and umask
chmod() changes permission bits. umask() defines which requested permissions are removed when new objects are created. Because umask is process-global state, changing it in a multithreaded application is risky.
import os
os.chmod("new.txt", 0o600)
Do not use chmod(0o777) as a universal fix. Grant only what is required and account for ACLs, container policies, mounted volumes, and the process identity.
User, group, and process identity
Functions such as getpid(), getppid(), getuid(), geteuid(), getgid(), and getegid() describe the current process. Real and effective identities may differ in specially privileged programs.
import os
print({
"pid": os.getpid(),
"uid": os.getuid(),
"euid": os.geteuid(),
"gid": os.getgid(),
})
To map IDs to account names, see Python pwd and Python grp. Those account databases are not authentication mechanisms.
Environment variables
At the POSIX level, the environment is a collection of keys and values attached to a process. In Python, prefer os.environ, a mutable mapping whose changes are inherited by later child processes.
import os
mode = os.environ.get("APP_MODE", "development")
os.environ["APP_CHILD_FLAG"] = "1"
Do not log the entire environment. It may contain tokens, passwords, private endpoints, and filesystem paths. Validate values before using them in commands or filenames.
Creating processes
POSIX exposes primitives such as fork(), execve(), waitpid(), and _exit(). They are powerful but require detailed handling of threads, buffers, locks, and inherited descriptors.
For launching programs, prefer subprocess. It models arguments, redirection, exit codes, timeouts, and descriptor closing more safely.
import subprocess
result = subprocess.run(
["uname", "-s"],
check=True,
capture_output=True,
text=True,
timeout=5,
)
print(result.stdout.strip())
Avoid shell=True with untrusted data. Pass arguments as a list.
Fork and threads
After fork(), only the calling thread remains in the child. Locks held by vanished threads may stay permanently locked. Database clients, loggers, TLS libraries, and memory allocators can also contain inconsistent state.
Modern applications should use subprocess or an appropriate multiprocessing start method. Avoid complex Python work between fork() and exec() in a multithreaded process.
Inherited descriptors
An open descriptor can leak into a child and keep files, sockets, or pipes alive. Python creates many descriptors as non-inheritable by default, but native extensions and explicit settings can change that behavior.
Use os.get_inheritable(), os.set_inheritable(), and subprocess options deliberately. Python fcntl covers Unix descriptor flags and locks.
Directory-relative operations
Many functions accept dir_fd. Instead of concatenating path strings, open a trusted directory and resolve names relative to that descriptor. This can reduce race conditions and make restricted operations easier to reason about.
import os
base_fd = os.open("data", os.O_RDONLY | os.O_DIRECTORY)
try:
fd = os.open("item.txt", os.O_RDONLY, dir_fd=base_fd)
try:
print(os.read(fd, 100))
finally:
os.close(fd)
finally:
os.close(base_fd)
Not every function or platform supports dir_fd. Inspect os.supports_dir_fd.
Synchronization and durability
fsync() asks the system to flush data and associated metadata for a descriptor. For critical replacement, write a temporary file on the same filesystem, flush it, rename it atomically, and, where required, flush the directory.
A successful call does not guarantee protection from every hardware or cache failure. Durability depends on the filesystem, storage device, mount options, and infrastructure.
Errors and OSError
System-call failures appear as subclasses of OSError, including FileNotFoundError, PermissionError, FileExistsError, and IsADirectoryError.
from pathlib import Path
try:
data = Path("config.ini").read_text(encoding="utf-8")
except FileNotFoundError:
data = ""
except PermissionError as error:
raise RuntimeError("cannot read config.ini") from error
Catch the narrowest exception that enables a meaningful response. Do not suppress unexpected failures with except OSError: pass.
Resource limits
POSIX calls may fail because the process has exhausted descriptors, memory, disk space, or process limits. Python resource explains how to inspect and set several Unix limits.
Handle failures, set timeouts, and cap input volume. A directory walker or batch processor should close resources progressively rather than keeping everything open.
Test on real systems
Containers do not reproduce every difference among macOS, Linux, and BSD. Test permissions, symlinks, byte paths, read-only mounts, network filesystems, full disks, different UIDs, umask behavior, signals, and descriptor inheritance.
Run tests with reduced privileges as well. Administrator-only testing can hide permission mistakes and unsafe assumptions.
Common mistakes
Frequent errors include importing posix when os is sufficient, assuming every function exists, forgetting to close descriptors, mixing text and bytes paths, following symlinks unintentionally, changing process-wide umask from several threads, using fork() unsafely, and catching OSError without diagnostics.
Conclusion
posix exposes the Unix system-call layer used by Python, but os should remain the default interface for most projects. Choose high-level APIs when they communicate intent more clearly, and use descriptors, flags, and directory-relative operations when precise control is genuinely needed.
Validate capabilities, close resources, minimize privilege, and treat shared paths as mutable. Consult the official posix documentation and the os documentation.
