Walking directory trees is common in automation, backups, auditing, cleanup, and indexing. For years, os.walk() was the standard tool. Modern Python also provides pathlib.Path.walk(), integrating traversal with the object-oriented pathlib API and returning the current directory as a Path.
This guide covers top-down and bottom-up traversal, directory pruning, error handling, symbolic links, file-size reports, safe deletion, deterministic ordering, concurrency hazards, and compatibility with older Python versions.
First Path.walk example
from pathlib import Path
root = Path("project")
for directory, dirnames, filenames in root.walk():
print("Directory:", directory)
for name in filenames:
path = directory / name
print(" File:", path)Each iteration returns the current directory as a Path, a list of subdirectory names, and a list of file names. Join names with the current directory to build full paths.
Why use pathlib?
Path keeps path joining, suffix handling, reading, writing, metadata, and relative-path operations in one consistent API.
for directory, _, filenames in root.walk():
for name in filenames:
path = directory / name
if path.suffix == ".py":
print(path.relative_to(root))This avoids repeatedly combining os.path.join(), os.path.splitext(), and other helpers. The guide to Python pathlib covers the broader API.
Top-down traversal
By default, the parent directory is yielded before its children.
for directory, dirnames, filenames in root.walk(top_down=True):
print(directory)Top-down mode allows code to modify dirnames and control recursion.
Skipping directories
Remove entries from the subdirectory list in place.
SKIP = {".git", ".venv", "node_modules", "__pycache__"}
for directory, dirnames, filenames in root.walk():
dirnames[:] = [name for name in dirnames if name not in SKIP]
for name in filenames:
print(directory / name)Slice assignment updates the exact list used by the walker. Writing dirnames = [...] only rebinds a local variable and does not prune traversal.
Deterministic order
Filesystem order should not be considered stable. Sort lists for reproducible output.
for directory, dirnames, filenames in root.walk():
dirnames.sort()
filenames.sort()
for name in filenames:
print(directory / name)Sorting adds work but is valuable for tests, reports, archives, and deterministic generation.
Bottom-up traversal
With top_down=False, children are yielded before the parent.
for directory, dirnames, filenames in root.walk(top_down=False):
print(directory)This is appropriate for deletion because files and nested directories must be removed before their parent.
Safely deleting a tree
from pathlib import Path
def remove_tree(root: Path) -> None:
for directory, dirnames, filenames in root.walk(top_down=False):
for name in filenames:
(directory / name).unlink()
for name in dirnames:
(directory / name).rmdir()
root.rmdir()This example is destructive. Validate the root, refuse dangerous locations, and consider shutil.rmtree() for a mature implementation. Never delete from untrusted input without strict boundaries.
Error handling with on_error
Directory-listing errors can be sent to a callback.
import logging
logger = logging.getLogger(__name__)
def record_error(error: OSError) -> None:
logger.warning("Could not access %s: %s", error.filename, error)
for directory, dirnames, filenames in root.walk(on_error=record_error):
...When completeness matters, log, count, or propagate failures instead of silently skipping inaccessible areas.
Failing on the first error
def fail(error: OSError) -> None:
raise error
for item in root.walk(on_error=fail):
...An indexer may continue with warnings, while a security audit may need to fail if any directory cannot be inspected.
Symbolic links
Directory symlinks require care. Following them can create cycles or escape the expected tree.
for directory, dirnames, filenames in root.walk(follow_symlinks=False):
...Not following links is the safer default. If following is required, track visited real directories and impose boundaries.
Cycle detection
On compatible systems, a set of device and inode pairs can detect repeated directories.
visited: set[tuple[int, int]] = set()
for directory, dirnames, filenames in root.walk(follow_symlinks=True):
info = directory.stat()
key = (info.st_dev, info.st_ino)
if key in visited:
dirnames.clear()
continue
visited.add(key)Inode and link semantics vary across platforms, network filesystems, and mount points. Test on the deployment environment.
Finding files by extension
EXTENSIONS = {".py", ".toml", ".json"}
found: list[Path] = []
for directory, dirnames, filenames in root.walk():
dirnames[:] = [d for d in dirnames if d not in {".git", ".venv"}]
for name in filenames:
path = directory / name
if path.suffix.lower() in EXTENSIONS:
found.append(path)For a simple recursive pattern without pruning requirements, Path.rglob() may be shorter. walk() is better when directory control, ordering, and errors matter.
Calculating tree size
def total_size(root: Path) -> int:
total = 0
for directory, _, filenames in root.walk():
for name in filenames:
path = directory / name
try:
total += path.stat().st_size
except OSError:
continue
return totalThe result is not a filesystem snapshot. Files may change, appear, or disappear during traversal.
Yielding large files
def larger_than(root: Path, limit: int):
for directory, _, filenames in root.walk():
for name in filenames:
path = directory / name
try:
size = path.stat().st_size
except OSError:
continue
if size > limit:
yield path, sizeA generator avoids holding every result in memory.
Path.walk versus os.walk
The concepts are similar: top-down control, directory pruning, error callbacks, and symbolic-link options. The practical difference is the current-directory type and integration with pathlib.
import os
for directory, dirnames, filenames in os.walk("project"):
...Projects already using Path may prefer Path.walk. Libraries supporting older Python versions can continue using os.walk().
Version compatibility
Path.walk() was added in modern Python. A package supporting older releases can provide a fallback.
from pathlib import Path
import os
def walk_compatible(root: Path):
if hasattr(root, "walk"):
yield from root.walk()
return
for directory, dirnames, filenames in os.walk(root):
yield Path(directory), dirnames, filenamesDeclare the minimum supported version in pyproject.toml and package metadata.
Concurrent filesystem changes
Directory trees change while they are being inspected. A listed file may disappear before stat(); permissions may change; directories may be renamed. Catch OSError close to the operation that can fail.
Checking exists() first does not eliminate races. Another process can change the path between the check and the operation. Prefer attempting the operation and handling failure.
Untrusted paths and security
Resolve and validate a user-provided root against an allowed base. Symlinks and .. segments can escape the intended area.
base = Path("/srv/uploads").resolve()
target = (base / user_input).resolve()
if not target.is_relative_to(base):
raise ValueError("path outside allowed area")The exact policy must also account for symbolic links and concurrent replacement.
Common mistakes
- Rebinding dirnames instead of modifying it: use slice assignment.
- Trusting filesystem order: sort when deterministic output matters.
- Following symlinks without cycle detection: traversal may never end.
- Ignoring errors without a policy: completeness may be misleading.
- Deleting in top-down order: use bottom-up traversal.
- Assuming a consistent snapshot: the tree may change during the walk.
Complete example: project inventory
from dataclasses import dataclass
from pathlib import Path
@dataclass
class FileInfo:
path: Path
size: int
def inventory(root: Path) -> list[FileInfo]:
result: list[FileInfo] = []
skip = {".git", ".venv", "node_modules", "__pycache__"}
for directory, dirnames, filenames in root.walk():
dirnames[:] = sorted(d for d in dirnames if d not in skip)
for name in sorted(filenames):
path = directory / name
try:
info = path.stat()
except OSError:
continue
result.append(
FileInfo(
path=path.relative_to(root),
size=info.st_size,
)
)
return resultThe inventory skips heavy directories, produces stable order, stores relative paths, and tolerates files disappearing.
Conclusion
Path.walk() brings directory-tree traversal into pathlib. It supports pruning, ordering, error handling, bottom-up traversal, and Path-based operations.
The official Python Path.walk documentation explains parameters and symbolic-link behavior. Use it when fine-grained traversal control is required, handle concurrent filesystem changes, and keep a fallback strategy when supporting older Python versions.
