The shelve module stores Python objects in a persistent, dictionary-like file. It is useful when a script needs to keep a small amount of state between runs without installing a database server. The API feels familiar, but the implementation has important consequences for security, portability, concurrency, and long-term maintenance.
When shelve is a good fit
Use shelve for local scripts, prototypes, command-line utilities, small caches, personal automation, and short-lived desktop tools. You open a shelf, assign values to string keys, and close it. For web applications, multiple workers, shared data, complex queries, or critical records, choose SQLite, PostgreSQL, or another transactional database instead.
A basic example
import shelve
with shelve.open("app_data") as db:
db["user"] = {"name": "Ana", "level": 3}
db["theme"] = "dark"
with shelve.open("app_data") as db:
print(db["user"])
The with statement closes the shelf even when an exception occurs. Depending on the operating system and dbm backend, one logical shelf may produce several files with different extensions. Treat the value passed to open as a database base name rather than assuming there will be exactly one file.
How it works internally
shelve combines a key-value database from the dbm family with object serialization provided by pickle. Keys are strings. Values may be lists, dictionaries, tuples, class instances, and other picklable objects. This flexibility is convenient, but the stored data is not human-readable like JSON and must never be loaded from an untrusted source.
Updating mutable values
A common mistake is to retrieve a dictionary or list, mutate it, and expect the change to be saved automatically. The safest pattern is to assign the modified object back to the shelf.
with shelve.open("app_data") as db:
profile = db["user"]
profile["level"] += 1
db["user"] = profile
The writeback=True option caches accessed objects and writes them when the shelf closes. It makes some code shorter, but it can consume substantial memory and make closing unexpectedly slow.
with shelve.open("app_data", writeback=True) as db:
db["user"]["level"] += 1
Use writeback only for small datasets when its behavior is understood. Explicit reassignment is more predictable and usually easier to review.
Useful dictionary operations
A shelf supports familiar methods such as keys(), values(), items(), get(), and pop(), plus membership testing with in. Full iteration may be expensive because each value can require deserialization.
with shelve.open("app_data") as db:
db["counter"] = db.get("counter", 0) + 1
if "settings" in db:
print(db["settings"])
for key in db.keys():
print(key)
Open flags
The flag argument controls how the underlying database is opened. The default c opens for reading and writing and creates the database when needed. r opens an existing database as read-only. w opens an existing database for reading and writing. n creates a new empty database and discards the old one.
Prefer r in code that should only read data, because it avoids accidental creation or modification. Use n only for deliberate rebuilds, isolated tests, or reset commands.
Security rules
Never open a shelf uploaded by a user, downloaded from the internet, received by email, or modified by an untrusted account. Pickle data can execute code while being loaded. Store shelf files in a controlled directory with restrictive permissions and keep them outside public web folders.
shelve does not encrypt values. Passwords, tokens, private keys, and API credentials belong in a secret manager or another protected storage system. File permissions alone may not satisfy the security requirements of production systems.
Concurrency and integrity
shelve does not provide a portable multi-writer locking strategy. Two processes writing to the same shelf can corrupt the database. The safest rule is one writer at a time. If multiple processes or machines must share the data, use SQLite with appropriate transactions or a database service designed for concurrency.
A crash or power loss can also leave files inconsistent. Back up valuable data before migrations, write updates to a temporary database when practical, and test restoration rather than assuming a copied file will always work.
Portability and migrations
The dbm implementation differs across operating systems and Python installations. Files created on one computer may not open on another. Pickled objects also depend on compatible module paths and class definitions. Renaming a class or moving it to another module can make old values impossible to load without migration code.
For records that must survive for years, travel between platforms, or be read by other programming languages, use JSON, CSV, SQLite, or a documented database schema. shelve works best for controlled local persistence.
A small repository wrapper
from pathlib import Path
import shelve
class Repository:
def __init__(self, path: Path):
self.path = str(path)
def save(self, key: str, value) -> None:
with shelve.open(self.path) as db:
db[key] = value
def get(self, key: str, default=None):
with shelve.open(self.path) as db:
return db.get(key, default)
def delete(self, key: str) -> bool:
with shelve.open(self.path) as db:
if key not in db:
return False
del db[key]
return True
Wrapping storage behind a repository centralizes paths, validation, logging, and backups. It also makes a future move to SQLite easier because the rest of the application depends on your interface instead of calling shelve everywhere.
Testing safely
Use tempfile.TemporaryDirectory in tests so every test receives isolated storage and the files are removed automatically.
from pathlib import Path
from tempfile import TemporaryDirectory
with TemporaryDirectory() as folder:
repo = Repository(Path(folder) / "test_db")
repo.save("x", {"value": 10})
assert repo.get("x")["value"] == 10
Operational best practices
Use stable, documented keys. Validate values before writing. Always close the database with with. Avoid writeback for large collections. Keep only one writer. Add backups for valuable state. Provide an export command, preferably to JSON, and test that exports can be restored on another machine.
Alternatives
JSON is better for simple portable data. SQLite adds transactions, queries, indexes, and safer concurrent access. dbm stores bytes and leaves serialization to the application. SQLModel and SQLAlchemy are appropriate when the domain and query requirements grow.
Conclusion
shelve is a practical bridge between temporary in-memory dictionaries and a full database. It can remove boilerplate from small local projects, but it should not become an invisible critical dependency. Use it only with trusted files, understand pickle risks, restrict concurrency, and plan a migration when durability or scale becomes important.
Continue with Python zipfile, Python tempfile, Python tomllib, and Python copy. See the official shelve documentation and the pickle security warning.
Before adopting the module, document what will be stored, how long it must remain available, and how recovery will work. This small architectural decision prevents a prototype from becoming an unmonitored production dependency.
