The Python hashlib module provides a consistent interface to cryptographic hash functions including SHA-256, SHA-512, SHA-3, SHAKE, and BLAKE2. A hash transforms an arbitrary amount of input into a fixed-size digest. Small input changes produce very different results, making hashes useful for file integrity, content identification, cache keys, deduplication, and cryptographic protocols.
Security depends on choosing the correct primitive for the job. SHA-256 can verify a file, but plain SHA-256 is not appropriate for storing passwords. A hash without a secret key does not prove who created a message. This guide covers the main APIs, incremental hashing, large files, safe comparison, SHAKE, BLAKE2, PBKDF2, and scrypt.
How hashing works
A hash function accepts bytes and produces a digest. The same bytes always produce the same result. There is no normal decrypt operation that recovers the original message, but predictable inputs can still be guessed through brute force. One-way behavior therefore does not provide automatic secrecy.
import hashlib
message = "Academify".encode("utf-8")
hash_object = hashlib.sha256(message)
print(hash_object.digest())
print(hash_object.hexdigest())
digest() returns raw bytes. hexdigest() returns hexadecimal text suitable for manifests, databases, and text protocols. The Python binascii guide explains binary and hexadecimal representations in detail.
Choosing an algorithm
SHA-256 is a common interoperable choice for new integrity checks. SHA-512 may be required by a protocol. SHA-3 uses a different construction from SHA-2. BLAKE2 is fast and supports configurable digest lengths and keyed mode. SHAKE produces variable-length output.
MD5 and SHA-1 have known collision weaknesses. They still occur in legacy formats and non-adversarial identifiers, but should not protect signatures, certificates, or attacker-controlled content. The usedforsecurity=False parameter can make an explicitly non-security use available in restricted builds; it does not repair the algorithm.
print(sorted(hashlib.algorithms_guaranteed))
print("sha256" in hashlib.algorithms_available)
algorithms_guaranteed lists portable algorithms. algorithms_available may include additional names supplied by the OpenSSL library linked to the interpreter.
Incremental updates
You do not need to concatenate or load all data into memory. Multiple calls to update() are equivalent to hashing the concatenated bytes.
hasher = hashlib.sha256()
hasher.update(b"part one")
hasher.update(b"part two")
print(hasher.hexdigest())
This pattern is ideal for uploads, sockets, compressed streams, and large files. It follows the same memory-conscious approach described in reading giant files without freezing Python.
Hashing a file with SHA-256
from pathlib import Path
import hashlib
def sha256_file(path: Path, chunk_size: int = 1024 * 1024) -> str:
hasher = hashlib.sha256()
with path.open("rb") as file:
while chunk := file.read(chunk_size):
hasher.update(chunk)
return hasher.hexdigest()
print(sha256_file(Path("package.zip")))
Compare the result with a digest obtained through an authenticated channel. If both file and checksum come from the same compromised source, the comparison does not prove authenticity. For higher-risk distribution, use HTTPS and a digital signature.
Compressed files may also contain internal checksums, but those usually detect accidental corruption. The guide to creating ZIP files with Python explains that distinction.
Using file_digest()
Since Python 3.11, hashlib.file_digest() simplifies binary file hashing and may use optimized I/O paths.
with open("package.zip", "rb") as file:
digest = hashlib.file_digest(file, "sha256")
print(digest.hexdigest())
Assume that the file position and internal state are unknown after the call. In Python 3.14, a non-blocking file raises BlockingIOError; earlier behavior could accidentally add null bytes to the digest.
Comparing digests
A regular comparison is generally sufficient for a public file checksum. When comparing secret authentication values, use hmac.compare_digest(), which reduces timing differences caused by the first mismatching byte.
import hmac
expected = "a" * 64
calculated = sha256_file(Path("package.zip"))
if not hmac.compare_digest(expected, calculated):
raise ValueError("File digest does not match")
Constant-time comparison cannot fix an unauthenticated expected value or a weak protocol. The source of the expected digest remains part of the security model.
SHAKE variable-length output
shake_128 and shake_256 require an output length when requesting a digest.
identifier = hashlib.shake_256(b"record-123").hexdigest(20)
print(identifier) # 40 hexadecimal characters
Choose the size based on collision requirements and the threat model. Do not shorten digests arbitrarily when identifiers are derived from attacker-controlled content.
BLAKE2 and domain separation
BLAKE2b is optimized for 64-bit systems and returns between 1 and 64 bytes. BLAKE2s returns up to 32 bytes. The person argument provides personalization, ensuring that the same input used for different purposes produces unrelated digests.
file_hash = hashlib.blake2b(
b"content",
digest_size=32,
person=b"Files-v1",
).hexdigest()
block_hash = hashlib.blake2b(
b"content",
digest_size=32,
person=b"Blocks-v1",
).hexdigest()
assert file_hash != block_hash
BLAKE2 also supports a native keyed mode. It can authenticate short application messages efficiently, although standard HMAC is often preferred when interoperability and conventional security review are priorities.
Why plain SHA-256 is wrong for passwords
General-purpose hashes are intentionally fast. That is excellent for files but allows attackers to test huge password dictionaries quickly. Password storage needs a slow, tunable, salted algorithm such as Argon2id, scrypt, bcrypt, or PBKDF2.
The Python password hashing guide covers the full authentication workflow. Never copy a simple file-hashing example into a password database.
PBKDF2 with a random salt
import os
import hashlib
salt = os.urandom(16)
iterations = 600_000
password = "long password".encode("utf-8")
derived = hashlib.pbkdf2_hmac(
"sha256",
password,
salt,
iterations,
)
Store the algorithm, iteration count, salt, and derived value. Calibrate the cost on production-class hardware and impose a sensible maximum password length before derivation.
scrypt and memory cost
hashlib.scrypt() adds memory cost, making specialized parallel attacks more expensive.
derived = hashlib.scrypt(
password,
salt=salt,
n=2**14,
r=8,
p=1,
dklen=32,
)
Parameters must be benchmarked and stored with the resulting value. Handle invalid parameters and memory errors explicitly rather than silently lowering the security settings.
Content IDs, caching, and deduplication
Digests are useful as content identifiers and cache keys. Include every option that changes the result, and add a version marker so a future format change does not reuse old entries.
def cache_key(url: str, language: str) -> str:
canonical = f"v1\n{language}\n{url}".encode("utf-8")
return hashlib.sha256(canonical).hexdigest()
When hashing API data, define canonical encoding and ordering. The guide to consuming REST APIs in Python provides additional validation and transport practices.
Common mistakes
Frequent errors include passing str without encoding, using MD5 or SHA-1 for security, confusing hashing with encryption, using plain SHA-256 for passwords, comparing authentication tags with ==, trusting a checksum from the same insecure channel, and reading entire files into memory.
Best practices
Use SHA-256 or the modern algorithm required by the protocol. Stream large input. Record the scheme and version with the digest. Separate integrity from authenticity. Use HMAC or signatures when origin matters. Use a salted, slow KDF for passwords. Do not invent custom cryptographic constructions.
Conclusion
hashlib supports basic file digests, SHA-3, SHAKE, BLAKE2, PBKDF2, and scrypt through a compact interface. The API is easy; the security decision is contextual. A checksum can detect a change, an authenticated message needs a key, a password needs expensive derivation, and a trusted download needs an expected digest obtained authentically.
See the official hashlib documentation and NIST FIPS 180-4.
