Python difflib: Compare Text and Files

Comparing two versions of text is useful in reviews, audits, migrations, tests, synchronization, and command-line tools. Instead of only knowing that content differs, applications often need to show which lines were removed, added, or changed. The Python difflib module provides sequence matching, human-readable deltas, similarity suggestions, and HTML reports without external dependencies.

This guide covers SequenceMatcher, get_close_matches(), unified_diff(), context_diff(), ndiff(), restore(), and HtmlDiff. It complements our articles about Python lists, slicing, temporary files, collections, and reading text files.

What difflib compares

difflib works with sequences whose elements are hashable. This includes strings, lists of lines, and token sequences.

from difflib import SequenceMatcher

old = "Python is simple"
new = "Python is very simple"

matcher = SequenceMatcher(None, old, new)
print(matcher.ratio())

The algorithm finds long contiguous matching blocks and applies the same idea recursively to the remaining portions. It favors matches that look natural to people, but it does not promise a minimal edit script.

Similarity with ratio()

ratio() returns a value between zero and one. Values close to one indicate strong similarity.

similarity = SequenceMatcher(None, "configuration", "configuraton").ratio()
print(round(similarity, 3))

The score is not a probability and has no universal threshold. Calibrate a cutoff with real examples and measure false positives and false negatives.

Argument order can matter

The official difflib documentation warns that ratio() can change when the sequences are reversed.

print(SequenceMatcher(None, "tide", "diet").ratio())
print(SequenceMatcher(None, "diet", "tide").ratio())

If a business rule needs a symmetric score, compute both directions or choose a metric designed to be symmetric.

quick_ratio() and real_quick_ratio()

These methods return faster upper bounds for the complete ratio.

matcher = SequenceMatcher(None, "abcd", "bcde")
print(matcher.real_quick_ratio())
print(matcher.quick_ratio())
print(matcher.ratio())

In a large candidate search, use the estimates to reject obvious mismatches and calculate the full score only for survivors.

Finding matching blocks

get_matching_blocks() reports positions and sizes of equal blocks.

matcher = SequenceMatcher(None, "abxcd", "abcd")
for block in matcher.get_matching_blocks():
    print(block)

The final block is always a zero-size sentinel. This API is useful for custom highlighting and visualizations.

Transformations with get_opcodes()

get_opcodes() describes how to transform the first sequence into the second.

a = "qabxcd"
b = "abycdf"
matcher = SequenceMatcher(None, a, b)

for tag, i1, i2, j1, j2 in matcher.get_opcodes():
    print(tag, a[i1:i2], b[j1:j2])

Tags are equal, replace, delete, and insert. Structured opcodes are easier to consume in a UI than formatted diff text.

Comparing many inputs against one reference

SequenceMatcher caches details about the second sequence. Set a shared reference once when many inputs are compared with the same catalog value.

matcher = SequenceMatcher(None)
matcher.set_seq2("configuration")

for entry in ["configuraton", "config", "configurations"]:
    matcher.set_seq1(entry)
    print(entry, matcher.ratio())

This can reduce repeated work in typo correction and record normalization.

Suggestions with get_close_matches()

get_close_matches() returns the best candidates above a cutoff.

from difflib import get_close_matches

commands = ["install", "inspect", "initialize", "interrupt"]
print(get_close_matches("instal", commands, n=3, cutoff=0.6))

It is useful for CLIs, field validation, and “did you mean?” messages. It is not semantic search because it compares sequence form rather than meaning.

The autojunk heuristic

For sequences with at least 200 elements, the automatic heuristic treats highly repeated items as popular and reduces their role in matching.

matcher = SequenceMatcher(None, seq_a, seq_b, autojunk=False)

Disable autojunk when repetition is meaningful, such as DNA, structured logs, or code lists. Benchmark the change because disabling it can increase cost.

Custom junk elements

The first constructor argument can mark elements that should not become synchronization anchors.

matcher = SequenceMatcher(
    lambda ch: ch in " \t",
    "name = value",
    "name=value",
)

Junk guides matching but does not erase differences from the output. If whitespace should be completely ignored, normalize text first and preserve originals for display.

Unified diffs

unified_diff() produces the format commonly used by patches and source reviews.

from difflib import unified_diff

before = "line 1\nold line\nline 3\n".splitlines(keepends=True)
after = "line 1\nnew line\nline 3\n".splitlines(keepends=True)

diff = unified_diff(
    before,
    after,
    fromfile="before.txt",
    tofile="after.txt",
)
print("".join(diff))

The function returns a generator. Convert it to a list only if the output must be reused or counted.

Preserving line endings

Use splitlines(keepends=True) for file comparison. Diff control lines include terminators by default so they work with writelines().

For inputs without line endings, pass lineterm="" to avoid inconsistent output.

Context diffs

context_diff() presents changed clusters in a before/after style.

from difflib import context_diff

for line in context_diff(before, after, fromfile="a", tofile="b", n=2):
    print(line, end="")

The n parameter controls how many unchanged lines surround a modification.

Detailed output with ndiff()

ndiff() prefixes every line:

• - : unique to the first input;
• + : unique to the second;
• : shared;
• ? : visual guidance for intraline differences.

from difflib import ndiff

result = list(ndiff(before, after))
print("".join(result))

Lines beginning with ? were not in either original and can be confusing when tabs are present.

Restoring text with restore()

An ndiff() delta can reconstruct either source.

from difflib import restore

original = "".join(restore(result, 1))
modified = "".join(restore(result, 2))

assert original == "".join(before)
assert modified == "".join(after)

This restoration works for Differ-style output, not arbitrary unified patches.

Differ for line-by-line comparison

The Differ class exposes similar output with optional line and character filters.

from difflib import Differ

comparator = Differ()
result = comparator.compare(before, after)
print("".join(result))

Differ does not claim to produce minimal deltas. Local matches are often easier for humans to understand than accidental distant synchronization.

Side-by-side reports with HtmlDiff

HtmlDiff generates a table or complete document with interline and intraline highlighting.

from difflib import HtmlDiff

html = HtmlDiff(wrapcolumn=80).make_file(
    before,
    after,
    fromdesc="Previous version",
    todesc="New version",
    context=True,
    numlines=3,
)

with open("comparison.html", "w", encoding="utf-8") as file:
    file.write(html)

This works well for editorial review and audit reports.

HtmlDiff security

fromdesc and todesc are interpreted as unescaped HTML. Escape user-provided values.

from html import escape

title = escape(user_supplied_name)

The generated report also contains document fragments and should be handled as potentially sensitive data.

Comparing bytes

diff_bytes() helps when encodings are unknown or inconsistent.

from difflib import diff_bytes, unified_diff

a = [b"old line\n"]
b = [b"new line\n"]

result = diff_bytes(unified_diff, a, b)
print(b"".join(result))

For known encodings, correct decoding remains preferable because it gives meaningful character-level differences.

difflib versus filecmp

difflib explains content differences. The filecmp module answers whether files or directory trees appear equal and can use metadata for a shallow comparison.

Use filecmp.cmp(..., shallow=False) to confirm content equality, then generate a textual report with difflib when needed. dircmp identifies common, unique, and differing files in directory trees.

Normalization before comparison

Depending on the goal, normalize Unicode, line endings, whitespace, case, and volatile fields.

import unicodedata


def normalize(text: str) -> str:
    text = unicodedata.normalize("NFC", text)
    return "\n".join(line.rstrip() for line in text.splitlines())

Over-normalization can hide important changes. Offer strict and tolerant comparison modes when appropriate.

Comparing JSON and structured data

Raw JSON comparison may report indentation and key-order changes. Parse it, serialize with stable ordering, and compare the normalized form.

import json

normalized = json.dumps(obj, sort_keys=True, indent=2, ensure_ascii=False)

For deep semantic changes, a structure-aware diff tool may be more appropriate.

Performance and limits

SequenceMatcher can have quadratic worst-case cost. Huge files and highly repetitive sequences require size, memory, and execution-time limits.

Do not accept unlimited documents in a synchronous API. Consider specialized external tools, background job infrastructure, sampling, or streaming approaches for large inputs.

Testing a similarity rule

def similar(a: str, b: str, threshold: float = 0.8) -> bool:
    return SequenceMatcher(None, a, b).ratio() >= threshold

assert similar("install", "instal")
assert not similar("install", "remove")

Build a labeled dataset of real examples to calibrate the threshold. Test empty strings, Unicode, repetition, whitespace, and reversed arguments.

Common mistakes

• Treating ratio() as a probability.
• Using difflib as semantic search.
• Ignoring argument-order effects.
• Comparing files without preserving line endings.
• Rendering unescaped descriptions in HtmlDiff.
• Disabling autojunk without benchmarking.
• Loading unlimited files into memory.
• Expecting a minimal edit sequence.

Best practices

• Choose character, token, or line granularity.
• Normalize only differences irrelevant to the domain.
• Calibrate cutoffs with real data.
• Use unified diffs for tools and HTML for people.
• Escape metadata inserted into HTML.
• Define size and time limits.
• Use filecmp first when equality is enough.
• Test Unicode, whitespace, and repeated sequences.

Conclusion

The Python difflib module turns sequence comparison into reports useful to people and programs. SequenceMatcher provides blocks, opcodes, and similarity scores; get_close_matches() suggests alternatives; and diff functions produce unified, contextual, detailed, and HTML formats.

Results improve when an application defines what counts as a meaningful change, chooses the correct granularity, and limits large inputs. With controlled normalization, safe HTML handling, and threshold tests, difflib provides a strong foundation for reviewers, validators, CLIs, and audit tools.