Monitoring folders in real time is one of the most powerful ways to build automation systems that react instantly to file system changes. Imagine a scenario where, the moment you save an image in a folder, a Python script automatically resizes it, or when a new PDF report appears in a directory, it gets sent by email without any human intervention. This ability to “watch” the file system transforms ordinary scripts into intelligent workflow tools. The watchdog library makes this possible in Python with minimal code and negligible resource consumption. This guide covers everything from installation to a production-ready monitoring script.
What Is the Watchdog Library and Why Use It?
The watchdog library is a cross-platform Python API for monitoring file system events. Unlike older polling methods where the program repeatedly asked the operating system “has anything changed?” every second (which wastes CPU cycles), watchdog uses native OS hooks. According to the official watchdog page on PyPI, it taps into platform-specific mechanisms such as inotify on Linux, FSEvents on macOS, and ReadDirectoryChangesW on Windows. This means the OS notifies the library the instant an event occurs, keeping resource usage extremely low while maintaining an immediate response.
Beyond efficiency, watchdog distinguishes between several event types: file creation, content modification, file moves between directories, and permanent deletion. This granularity allows you to build precise automation that reacts differently to each type of change, which is exactly the kind of intelligent behavior that separates a well-designed Python automation script from a naive loop.
Setting Up the Environment
Before writing any code, install the watchdog library. Using a dedicated Python virtual environment is strongly recommended to keep project dependencies isolated. Open your terminal and run:
pip install watchdogAfter installation you will have access to two core components: the Observer class, which runs in a background thread and watches the chosen directory, and the FileSystemEventHandler class, which you subclass to define what should happen when each type of event fires. If you encounter path-related issues when accessing detected files, knowing how to manipulate file paths with pathlib will help you build reliable cross-platform paths.
Understanding the Observer and Handler Architecture
Think of the Observer as a security guard patrolling a door, and the Handler as the protocol that guard follows: “If someone enters, log their name; if someone leaves, note the time.” In Python terms, the Observer runs continuously in the background, while the Handler contains the business logic that executes each time an event is detected.
You customize behavior by inheriting from FileSystemEventHandler and overriding four key methods: on_created, on_modified, on_deleted, and on_moved. Each receives an event object containing useful information like the file path and a boolean indicating whether the event affected a directory or a regular file.
Creating a Custom Event Handler
Here is the basic handler structure. Each event method receives an event object, and event.src_path always contains the full path of the affected file:
from watchdog.events import FileSystemEventHandler
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
print(f"File modified: {event.src_path}")
def on_created(self, event):
print(f"New file detected: {event.src_path}")At this stage, it is common to encounter encoding errors when file names contain special characters. The guide on resolving UTF-8 encoding errors in Python covers the most common fixes so your script does not stop unexpectedly when scanning folders with accented file names.
Implementing the Main Observer Loop
With the handler ready, configure the engine that keeps the script running. The Observer uses threads internally so it does not block your program’s main execution. You specify which folder to watch and whether it should also watch subdirectories recursively:
import time
from watchdog.observers import Observer
observer = Observer()
handler = MyHandler()
observer.schedule(handler, path="path/to/folder", recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()The try/except block captures the stop command (Ctrl+C). Without it, the Observer thread could keep running in the background even after you close the terminal. Using the Python time module with sleep(1) is essential to prevent the while True loop from consuming 100% of your CPU unnecessarily.
Filtering Specific File Types
In most real projects you do not want to monitor all files, only specific extensions like .json, .csv, or .txt. You can implement this filtering logic directly inside the handler using a simple conditional check. For example, if a .csv is detected, the script can immediately trigger a data cleaning pipeline using Pandas. If it is a log file, it can scan for error keywords and send an alert through a Telegram bot:
def on_created(self, event):
if event.src_path.endswith(".csv"):
print(f"New CSV file ready for processing: {event.src_path}")
elif event.src_path.endswith(".pdf"):
print(f"New PDF document detected: {event.src_path}")Handling Permission Errors and Access Issues
When monitoring system folders or network directories, your script can run into access problems. A PermissionError in Python typically occurs when Python tries to read a file that is still being written by another program, or when the user lacks administrative privileges on the watched directory.
To handle this gracefully, add a small delay before accessing a newly created file, and wrap file access logic in try and except blocks. You should also filter out temporary system files that are created and deleted within milliseconds (they often start with a dot or tilde character).
Best Practices: Logging and Performance
In production systems, printing to the console with print() is not sufficient. Use the Python logging module to write the event history to a file. This lets you audit what happened overnight, for example if a critical file mysteriously disappeared during off-hours.
For performance, if you are monitoring a folder where thousands of files are created simultaneously, sequential processing can become a bottleneck. In those advanced scenarios, send detected files to a processing queue or use separate threads for each heavy task, ensuring the Observer never stops listening for new events while processing is underway.
Complete Project Code
Here is the full unified script. It monitors creation, modification, deletion, and movement of files in the folder where it runs. Save it as monitor.py and run it directly from your terminal:
import time
import os
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class EventManager(FileSystemEventHandler):
"""Defines the behavior triggered by file system changes."""
def on_created(self, event):
kind = "Folder" if event.is_directory else "File"
print(f"[CREATED] {kind}: {event.src_path}")
def on_modified(self, event):
kind = "Folder" if event.is_directory else "File"
print(f"[MODIFIED] {kind}: {event.src_path}")
def on_deleted(self, event):
kind = "Folder" if event.is_directory else "File"
print(f"[DELETED] {kind}: {event.src_path}")
def on_moved(self, event):
print(f"[MOVED] From {event.src_path} to {event.dest_path}")
if __name__ == "__main__":
# Set the path to monitor (current directory by default)
path_to_monitor = "."
event_handler = EventManager()
observer = Observer()
observer.schedule(event_handler, path_to_monitor, recursive=False)
print(f"Starting monitoring at: {os.path.abspath(path_to_monitor)}")
print("Press Ctrl+C to stop.")
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
print("nStopping monitor...")
observer.stop()
observer.join()This script is the perfect foundation for advanced automations. You can expand it to trigger an automatic backup with Python whenever an important file is modified, or to send real-time alerts when unexpected deletions occur in a critical business directory.
Frequently Asked Questions
Does watchdog work on network folders (NAS)?
Yes, but it depends on the protocol. On SMB/CIFS network mounts, watchdog generally works but may experience delays or missed events if the network server does not propagate notifications to the local OS correctly.
How do I monitor only PDF files?
Inside the on_created method, add a check: if event.src_path.endswith(".pdf"):. Only files matching that extension will trigger the subsequent logic, and all others will be silently ignored.
Does the script use a lot of RAM?
No. Watchdog is extremely lightweight because it relies on OS-level notifications rather than polling. Memory consumption depends far more on what you do with the detected files than on the monitoring itself.
Can I run this script as a Windows service?
Yes. You can use the pywin32 library to register the script as a Windows service, or convert it to an executable using PyInstaller and schedule it to run at startup using Windows Task Scheduler.
Does watchdog detect when a file is opened for reading?
No. Watchdog focuses on structural changes such as writes, creations, deletions, and moves. Read-only access does not trigger any events.
What happens when I delete a folder containing subfolders?
If recursive=True is set, watchdog fires events for every item inside the deleted folder. With recursive=False, only the deletion of the parent directory is reported.
How do I avoid duplicate events when saving a file?
Some editors write a temporary file before overwriting the original, triggering multiple modification events. A common solution is to compare the file size or use a small delay, processing only after a quiet period with no new events for the same file.
Is it possible to monitor the entire computer?
Technically yes, but it is not recommended. Monitoring a root drive like C: or / will generate thousands of events per second from system logs and temporary files, which will overload your processing logic immediately.
