Python pkgutil: Discover Packages

Python pkgutil provides utilities for the import system and package support. It can list available modules, recursively walk subpackages, inspect importers, resolve a textual name to an object, and extend the search path of a package distributed across multiple directories.

These features are useful for plugin systems, diagnostics, documentation generators, and environment audits. They also require care: some operations import packages to discover children, and importing a package executes its top-level code. Legacy resource APIs accept paths and should only receive trusted values.

Understand ModuleInfo

pkgutil.ModuleInfo is a named tuple containing the finder, module name, and a boolean indicating whether the result is a package.

from pkgutil import iter_modules

for info in iter_modules():
    print(info.name, info.ispkg, info.module_finder)

The object is a summary. iter_modules() does not import every discovered module, which makes it useful for quick listings.

List top-level modules

Without a path, iter_modules() scans the modules visible through sys.path.

import pkgutil

names = sorted(info.name for info in pkgutil.iter_modules())
print(names[:20])

The result depends on the active virtual environment, Python installation, and custom paths. Record that context in reports. Python sysconfig helps identify installation directories.

List modules inside a package

Pass a package’s __path__ and a prefix to restrict discovery.

import pkgutil
import my_app

for info in pkgutil.iter_modules(
    my_app.__path__,
    my_app.__name__ + '.',
):
    print(info.name)

The prefix produces fully qualified names such as my_app.plugins.csv.

Walk packages recursively

walk_packages() recursively discovers children. It must import packages to obtain their __path__.

import pkgutil
import my_app

for info in pkgutil.walk_packages(
    my_app.__path__,
    my_app.__name__ + '.',
):
    print(info.name, info.ispkg)

This side effect matters. A package initializer may open connections, read configuration, register handlers, or fail because an external service is unavailable. Do not walk unknown package trees in a privileged process.

Handle walk errors

The onerror callback receives the name of a package that failed during import.

errors = []

def record_error(name):
    errors.append(name)

for info in pkgutil.walk_packages(
    my_app.__path__,
    my_app.__name__ + '.',
    onerror=record_error,
):
    pass

Without a callback, ImportError is generally ignored while other exceptions propagate. Record failures without masking required components.

Discover plugins safely

A system can dedicate one package to plugins.

import pkgutil
import my_app.plugins

candidates = []
for info in pkgutil.iter_modules(
    my_app.plugins.__path__,
    'my_app.plugins.',
):
    leaf = info.name.rsplit('.', 1)[-1]
    if leaf.isidentifier():
        candidates.append(info.name)

Discovery is not authorization. Apply an allowlist, check metadata and compatible versions, and import only after validation.

Inspect path importers

get_importer(path_item) returns the finder associated with one path entry. Newly created finders are cached in sys.path_importer_cache.

import pkgutil

finder = pkgutil.get_importer('/project/plugins')
print(type(finder).__name__)

If path hooks change at runtime, the relevant cache may need invalidation. Libraries should avoid global import-hook changes unless they own the application lifecycle.

Iterate importers

iter_importers(fullname) yields finders capable of searching for a module name.

for finder in pkgutil.iter_importers('my_app.plugins.csv'):
    print(finder)

When the name belongs to a package, its parent package may be imported as a side effect. Apply the same caution used with walk_packages().

Extend a package path

extend_path() is a historical mechanism for distributing parts of one logical package across directories.

# my_namespace/__init__.py
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

Native namespace packages are usually preferred in new projects that do not require __init__.py, but legacy ecosystems may still depend on this approach.

Trust implications of .pkg files

extend_path() also reads matching *.pkg files. Their entries are accepted as declared, even if a path does not currently exist.

Treat these files as trusted configuration. Anyone who can modify them may influence import locations. Protect permissions and never generate them from untrusted input.

Resolve an object by name

resolve_name() turns a string into a module, class, function, or nested attribute.

from pkgutil import resolve_name

function = resolve_name('my_app.tasks:execute')
function()

The colon form is explicit: the left side is the module to import and the right side is the object hierarchy. The legacy dotted-only form may require repeated import attempts.

Validate names before resolving

Resolving a name imports code and returns an arbitrary object. Do not accept user-provided targets freely.

TARGETS = {
    'report': 'my_app.tasks:generate_report',
    'cleanup': 'my_app.tasks:clean_temp',
}

function = resolve_name(TARGETS[action])
if not callable(function):
    raise TypeError('Configured target is not callable')

An allowlist keeps the command surface explicit.

Read resources with get_data()

pkgutil.get_data(package, resource) requests bytes through the package loader.

import pkgutil

data = pkgutil.get_data('my_app', 'data/config.json')
if data is None:
    raise FileNotFoundError('Resource unavailable')

It works with loaders implementing get_data, including some ZIP packages. Namespace packages may not support it.

Prevent path traversal

The documentation warns that get_data() is intended for trusted input. Parent components and absolute paths may reach files outside the expected resource area, depending on the loader.

RESOURCES = {
    'default': 'data/default.json',
    'theme': 'data/theme.css',
}

content = pkgutil.get_data('my_app', RESOURCES[key])

Use known names, allowed extensions, and a fixed mapping. For new code, prefer Python importlib.resources.

pkgutil versus importlib.resources

pkgutil.get_data() remains useful for legacy loaders. importlib.resources provides Traversable objects, text helpers, directory support, and temporary filesystem contexts through a more structured API.

Choose importlib.resources for new implementations unless compatibility requires pkgutil.

Separate discovery from loading

iter_modules() discovers candidates without importing every module. walk_packages() imports packages but not necessarily every final module.

Design four phases: discover, validate, authorize, then import. This reduces effects and makes audit logs clearer.

Combine pkgutil with modulefinder

Python modulefinder starts with a script and follows imports. Pkgutil starts with paths and lists available modules. The two views complement each other.

A plugin may be installed and visible to pkgutil but never referenced by the entry script. A dynamically loaded plugin may require an explicit manifest.

Discover modules inside zipapps

iter_modules() supports common file finders and zipimporter. It can therefore discover modules within a Python zipapp when the finder implements the required interface.

Test the packaged artifact because behavior may differ from the source directory.

Invalidate caches after changes

If plugins are installed while a process is running, invalidate import caches and rescan carefully.

import importlib
import sys

importlib.invalidate_caches()
sys.path_importer_cache.pop('/project/plugins', None)

Avoid installing packages concurrently with imports from other threads. Restart workers after plugin updates when possible.

Performance

Scanning all of sys.path may be expensive. Restrict searches to a known package path and use a prefix.

Interactive tools can cache results by environment version and invalidate the cache after updates.

Test package discovery

Create a temporary directory containing known modules and packages.

def test_plugin_listing(tmp_path):
    root = tmp_path / 'plugins'
    root.mkdir()
    (root / 'alpha.py').write_text('NAME = "alpha"\n')

    found = [
        info.name
        for info in pkgutil.iter_modules([str(root)])
    ]
    assert 'alpha' in found

Also test import failures, namespace packages, ZIP archives, and initializers with side effects.

Common mistakes

• Walking all of sys.path unnecessarily.
• Ignoring that walk_packages() imports packages.
• Trusting every discovered module as a plugin.
• Resolving user-provided object names.
• Passing free-form paths to get_data().
• Treating underscore filtering as access control.
• Keeping stale caches after plugin installation.

Best practices

• Restrict searches to known packages.
• Separate discovery, validation, and import.
• Use allowlists for plugins and targets.
• Prefer importlib.resources in new code.
• Protect .pkg files.
• Record import failures with context.
• Restart workers after major changes.

Conclusion

Python pkgutil provides practical tools for listing modules, walking packages, inspecting finders, resolving objects, and supporting distributed package paths.

Use these APIs with awareness of their effects. Importing packages runs code, resolving names loads objects, and resource paths can escape expected boundaries. Consult the official pkgutil documentation and the Python import system reference.