Deleting a directory tree looks simple until protected files, changing permissions, symbolic links, concurrent processes, or partial failures appear. shutil.rmtree() handles recursive deletion, while the onexc parameter gives your application a central place to process exceptions predictably. This is useful in cleanup scripts, automated tests, installers, data pipelines, deployment jobs, and applications that create temporary workspaces.
This guide explains how to use shutil.rmtree with onexc, how to understand the callback arguments, when a retry is reasonable, when an error should be logged, and when the operation must stop. The goal is not merely to remove folders, but to do it with safety, observability, and reproducible behavior.
What shutil.rmtree does
shutil.rmtree(path) deletes a directory and everything below it. Unlike Path.rmdir() or os.rmdir(), the directory does not need to be empty. That makes the function convenient and powerful, but also dangerous: an incorrect path can remove a large amount of data.
from shutil import rmtree
rmtree("build")
Before calling it, validate the target, avoid paths built directly from untrusted input, and never apply recursive deletion to a critical directory. For modern path traversal patterns, see Python pathlib.Path.walk and Python os.fwalk.
Why onexc matters
Several internal operations can fail during recursive deletion. A file may be read-only, an antivirus program may keep it open, permissions may change between listing and deletion, or another process may remove the item first. The onexc parameter receives a callback whenever an exception occurs.
from shutil import rmtree
def on_failure(function, path, error):
print(f"Failed in {function.__name__}: {path}: {error}")
rmtree("build", onexc=on_failure)
The callback receives the function that failed, the path involved, and the exception object. This lets you make focused decisions without scattering broad try/except blocks across the application.
onexc versus ignore_errors
ignore_errors=True suppresses failures during removal. It is simple, but it also removes visibility. The program may finish while files remain, and later steps can assume a clean state that does not actually exist.
rmtree("cache", ignore_errors=True)
onexc is better when you need to record, repair, classify, or re-raise failures. Ignoring errors can be acceptable for disposable cleanup where an incomplete result has no consequence, but it should not be the default for critical workflows.
Repairing a read-only file
A common case is a file that cannot be deleted because its permissions do not allow modification. A callback can change that permission and retry only the failed operation.
import os
import stat
from shutil import rmtree
def repair_permission(function, path, error):
if isinstance(error, PermissionError):
os.chmod(path, stat.S_IWRITE)
function(path)
return
raise error
rmtree("output", onexc=repair_permission)
This approach should be restricted to a directory tree owned by your application. Changing permissions on arbitrary targets can magnify the impact of a configuration error. The callback should repair only known situations and re-raise everything else.
Do not hide unknown exceptions
A callback that only prints and returns may leave the tree partially removed. In critical cleanup, re-raise errors that cannot be safely corrected.
def handle(function, path, error):
if isinstance(error, FileNotFoundError):
return
raise error
FileNotFoundError may be harmless when another worker already removed the item. A permission failure, filesystem error, invalid path, or unexpected operating-system condition can indicate a larger problem. For explicit API contracts, read Python dataclasses.KW_ONLY.
Logging useful context
Production code should usually replace print() with structured logging. Include the failing function, path, exception type, and execution identifier.
import logging
from shutil import rmtree
log = logging.getLogger(__name__)
def record(function, path, error):
log.error(
"failed to remove path",
extra={
"operation": function.__name__,
"path": path,
"error_type": type(error).__name__,
},
)
raise error
rmtree("artifacts", onexc=record)
Good logs help distinguish a transient lock from a permanent configuration problem. They also let a team measure how often cleanup remains incomplete.
Validate the target before deletion
A safe helper should resolve the path, compare it with an allowed root, and reject critical directories.
from pathlib import Path
from shutil import rmtree
ROOT = Path("/srv/my-app/work").resolve()
def delete_subdirectory(value):
target = (ROOT / value).resolve()
if target == ROOT or ROOT not in target.parents:
raise ValueError("Path is outside the allowed root")
rmtree(target)
This prevents traversal through values such as ../. It also stops an empty or malformed value from selecting the working root itself. For temporary workspace patterns, see Python tempfile.
Symbolic links and path attacks
Recursive filesystem operations require attention to symbolic links and race conditions. Modern Python implementations use additional protections on supported platforms, but your application must still control where targets come from and limit cleanup to directories it owns.
Do not run privileged cleanup against user-supplied paths. When a process has elevated permissions, a single validation mistake can reach files that would normally be protected.
Short and bounded retries
Some failures are temporary because another process still holds a file open. A retry policy should be small, bounded, and visible.
import time
from shutil import rmtree
class Handler:
def __init__(self, retries=2):
self.remaining = retries
def __call__(self, function, path, error):
if self.remaining and isinstance(error, PermissionError):
self.remaining -= 1
time.sleep(0.2)
function(path)
return
raise error
rmtree("cache", onexc=Handler())
Never turn the callback into an infinite loop. If the condition persists, fail clearly and preserve enough context for diagnosis.
Using rmtree in automated tests
Tests frequently create temporary directories and need to remove them even when an assertion fails. Prefer context managers and fixtures, with rmtree as a controlled cleanup mechanism.
from pathlib import Path
from tempfile import mkdtemp
from shutil import rmtree
folder = Path(mkdtemp())
try:
(folder / "data.txt").write_text("test", encoding="utf-8")
finally:
rmtree(folder)
When several resources must be closed dynamically, Python contextlib.ExitStack provides a flexible way to register cleanup callbacks.
Idempotent cleanup
An idempotent cleanup routine can run more than once without producing an invalid state. One basic strategy is to handle the case where the directory is already gone.
from pathlib import Path
from shutil import rmtree
def remove_if_present(path):
target = Path(path)
if target.exists():
rmtree(target)
The pre-check does not eliminate race conditions because the state may change immediately afterward. The callback can still tolerate FileNotFoundError when that outcome is acceptable.
Designing callbacks that are testable
Keep the callback small and deterministic. Separate policy from side effects. One function can decide whether an exception is recoverable, another can perform the repair, and a third can log the final failure. This makes unit testing easier and prevents a large callback from becoming an unstructured error handler.
def recoverable(error):
return isinstance(error, (FileNotFoundError, PermissionError))
Tests should cover a missing file, a read-only file, an unexpected exception, and a retry that still fails. Verify that unexpected errors are not swallowed.
Version compatibility
onexc is available in modern Python versions as a clearer exception callback for rmtree. Check the minimum Python version of your project before adopting it. A library that supports older environments may need a compatibility layer or a different callback strategy.
The official shutil.rmtree documentation describes the current signature, security details, and exception behavior. The pathlib documentation is a complementary reference for resolving and validating paths.
A safer cleanup helper
import logging
import os
import stat
from pathlib import Path
from shutil import rmtree
log = logging.getLogger(__name__)
def cleanup(path, root):
root = Path(root).resolve()
target = Path(path).resolve()
if target == root or root not in target.parents:
raise ValueError("Target is not allowed")
def onexc(function, value, error):
if isinstance(error, FileNotFoundError):
return
if isinstance(error, PermissionError):
os.chmod(value, stat.S_IWRITE)
function(value)
return
log.exception("incomplete cleanup", extra={"path": str(value)})
raise error
rmtree(target, onexc=onexc)
The helper validates the allowed root, tolerates concurrent disappearance, repairs one known permission case, and re-raises everything unexpected.
Operational safeguards
Consider adding a dry-run mode that lists intended targets without deleting them. In scheduled jobs, record the number of files before and after cleanup. For high-value data, move a directory to a quarantine area first and delete it only after validation. These patterns reduce the blast radius of a mistake and create a window for recovery.
Do not use recursive deletion as a substitute for retention policy. Define which directories may be deleted, how old they must be, and which process owns them. A robust cleanup job has both technical checks and business rules.
Best practices
Resolve the path before deletion. Restrict operations to a controlled root. Never pass external input directly. Re-raise unknown errors. Log enough context. Avoid infinite retries. Test protected files and already-removed directories. Confirm the Python version. Do not combine ignore_errors=True with an expectation of complete deletion.
Conclusion
shutil.rmtree is a direct tool for deleting directory trees, and onexc turns unpredictable failures into explicit policy decisions. With path validation, logs, narrow repairs, and re-raised exceptions, cleanup becomes far more reliable.
Start with a callback that records and re-raises. Add only recovery behaviors that you understand and can test. This prevents automation from hiding partially removed trees or expanding the impact of an incorrect path.
