Python Regex: Complete Regular Expressions Guide

Regular expressions describe text patterns. They can find dates in a document, extract identifiers from logs, normalize whitespace, validate a simple format, or split inconsistent input. Python provides regular-expression support through the standard-library re module.

This Python regex guide covers raw strings, literal characters, character classes, anchors, quantifiers, groups, alternation, search(), fullmatch(), findall(), finditer(), sub(), split(), flags, compiled patterns, named groups, greedy matching, and practical safety rules.

Regex is powerful, but it is not the best tool for every text problem. Plain string methods are clearer for fixed prefixes, exact replacements, and simple splitting. Structured parsers should handle formats such as JSON and HTML whenever possible.

Import the re Module

import re

No package installation is required. The official Python re documentation defines the complete API, and the official Regular Expression HOWTO provides a longer conceptual introduction.

Use Raw Strings for Patterns

Regex syntax uses backslashes, and Python string syntax also uses backslashes. Prefixing a pattern with r prevents most Python-level escape processing.

pattern = r"\d{4}-\d{2}-\d{2}"

Without a raw string, many patterns require doubled backslashes. Raw strings make the regex easier to read. A raw string still cannot end with a single backslash, so unusual edge cases may require ordinary escaped strings.

Literal Text and Metacharacters

Most characters match themselves. Metacharacters have special meanings:

Escape a metacharacter when it should be literal:

price_pattern = r"\$\d+\.\d{2}"

Use re.escape(text) when literal user-supplied text must be inserted into a pattern.

Character Classes

r"[abc]"       # a, b, or c
r"[a-z]"       # lowercase ASCII letter in this range
r"[^0-9]"      # character that is not an ASCII digit
r"\d"          # Unicode digit by default
r"\w"          # Unicode alphanumeric character or underscore
r"\s"          # whitespace
r"\D"          # not a digit
r"\W"          # not a word character
r"\S"          # not whitespace

\d is not always identical to [0-9]. In normal string patterns, \d can match Unicode decimal digits. Use [0-9] or the ASCII flag when a format specifically requires ASCII digits.

Search for the First Match

re.search() scans the string and returns the first match object or None.

import re

text = "Order 4832 is ready"
match = re.search(r"\d+", text)

if match:
    print(match.group())   # 4832
    print(match.start())   # starting index
    print(match.span())    # (start, end)

Always check whether a match exists before calling group().

match(), search(), and fullmatch()

• match() checks only at the beginning of the string.
• search() scans for a match anywhere.
• fullmatch() requires the entire string to match.

pattern = r"[A-Z]{3}-[0-9]{4}"

print(re.match(pattern, "ABC-1234 extra"))
print(re.search(pattern, "Plate: ABC-1234"))
print(re.fullmatch(pattern, "ABC-1234"))

Use fullmatch() for whole-value format validation. It expresses the intention more directly than manually adding start and end anchors.

Find Every Match

findall() returns matching strings, or tuples when the pattern contains multiple capturing groups.

text = "Tickets: AB-104, XY-220, CD-981"
codes = re.findall(r"[A-Z]{2}-[0-9]{3}", text)
print(codes)

Use finditer() when you need match objects, positions, named groups, or lazy iteration:

for match in re.finditer(r"[A-Z]{2}-[0-9]{3}", text):
    print(match.group(), match.span())

Quantifiers

r"a*"          # zero or more a characters
r"a+"          # one or more
r"a?"          # zero or one
r"a{3}"        # exactly three
r"a{2,}"       # at least two
r"a{2,5}"      # from two through five

Quantifiers apply to the token immediately before them. Group several tokens when the complete sequence should repeat:

r"(?:ab){2,4}"

(?:...) is a non-capturing group. It groups the pattern without adding a numbered capture.

Greedy and Non-Greedy Matching

Quantifiers are greedy by default and consume as much text as possible while still allowing a match.

text = "<b>first</b> and <b>second</b>"

print(re.findall(r"<b>.*</b>", text))
print(re.findall(r"<b>.*?</b>", text))

The second pattern uses *? for non-greedy repetition. This example demonstrates behavior, but a real HTML parser should process HTML. Regex alone cannot reliably model arbitrary nested markup.

Capture Groups

date_pattern = r"([0-9]{4})-([0-9]{2})-([0-9]{2})"
match = re.fullmatch(date_pattern, "2026-07-10")

if match:
    year, month, day = match.groups()
    print(year, month, day)

A pattern match verifies the shape, not whether the date is real. Use datetime to reject values such as an impossible month or day.

Use Named Groups

pattern = re.compile(
    r"(?P<year>[0-9]{4})-"
    r"(?P<month>[0-9]{2})-"
    r"(?P<day>[0-9]{2})"
)

match = pattern.fullmatch("2026-07-10")

if match:
    print(match.group("year"))
    print(match.groupdict())

Named groups make complex extraction code easier to understand than numeric positions.

Alternation

pattern = r"cat|dog|bird"
print(re.findall(pattern, "dog, cat, and bird"))

Group alternation when it is part of a larger pattern:

file_pattern = r"report\.(?:csv|json|xlsx)"

Replace Matches with sub()

text = "Too     many   spaces"
clean = re.sub(r"\s+", " ", text).strip()
print(clean)

The replacement can be a function:

def hide_digits(match):
    return "*" * len(match.group())


masked = re.sub(r"[0-9]+", hide_digits, "Card ending in 1234")
print(masked)

For fixed text replacement, str.replace() is usually simpler and faster to read.

Split with a Pattern

text = "red, green;blue | yellow"
parts = re.split(r"\s*[,;|]\s*", text)
print(parts)

Regex splitting is useful when several delimiters or inconsistent spaces are accepted.

Compile Reused Patterns

ticket_pattern = re.compile(r"[A-Z]{2}-[0-9]{3}")

for text in ["AB-123", "invalid", "XY-999"]:
    if ticket_pattern.fullmatch(text):
        print("Valid:", text)

Module-level functions also cache recent patterns, so compilation is mainly valuable for readability, reuse, attached methods, and explicit configuration.

Regex Flags

pattern = re.compile(
    r"^error:.*$",
    flags=re.IGNORECASE | re.MULTILINE,
)

text = "Info: started\nERROR: connection failed\nInfo: stopped"
print(pattern.findall(text))

• re.IGNORECASE or re.I: case-insensitive matching.
• re.MULTILINE or re.M: makes ^ and $ apply to each line.
• re.DOTALL or re.S: lets dot match newlines.
• re.VERBOSE or re.X: allows whitespace and comments in readable multi-line patterns.
• re.ASCII or re.A: restricts certain shorthand classes to ASCII behavior.

Write Readable Patterns with VERBOSE

simple_email_pattern = re.compile(
    r"""
    [A-Z0-9._%+-]+     # local part
    @
    [A-Z0-9.-]+        # domain
    \.
    [A-Z]{2,}           # final label
    """,
    re.IGNORECASE | re.VERBOSE,
)

This checks a common email-like format, not every valid email address and not whether the mailbox exists. For account registration, confirmation by sending a message is more meaningful than trying to encode the full email specification in one regex.

Validate a Format with fullmatch()

USERNAME_PATTERN = re.compile(r"[a-z][a-z0-9_]{2,19}")


def valid_username(value):
    return USERNAME_PATTERN.fullmatch(value) is not None


print(valid_username("ava_2026"))
print(valid_username("2bad"))

Regex validates the specified format only. Business rules, uniqueness, permissions, and database checks belong elsewhere.

Extract Structured Log Data

LOG_PATTERN = re.compile(
    r"^(?P<date>[0-9]{4}-[0-9]{2}-[0-9]{2})\s+"
    r"(?P<level>INFO|WARNING|ERROR)\s+"
    r"(?P<message>.+)$"
)

line = "2026-07-10 ERROR Database unavailable"
match = LOG_PATTERN.fullmatch(line)

if match:
    record = match.groupdict()
    print(record)

For application-generated output, the Python logging guide explains structured and persistent diagnostics.

Practical Project: Parse a Log File

The following program reads a text log, counts levels, and stores parsed records. It reports malformed lines with their original line numbers.

import re
from collections import Counter
from pathlib import Path

LOG_PATTERN = re.compile(
    r"^(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}T"
    r"[0-9]{2}:[0-9]{2}:[0-9]{2})\s+"
    r"(?P<level>DEBUG|INFO|WARNING|ERROR|CRITICAL)\s+"
    r"(?P<message>.+)$"
)


def parse_log_line(line):
    match = LOG_PATTERN.fullmatch(line.strip())

    if match is None:
        return None

    return match.groupdict()


def parse_log_file(path):
    records = []
    invalid_lines = []
    level_counts = Counter()

    with Path(path).open("r", encoding="utf-8", errors="replace") as file:
        for line_number, line in enumerate(file, start=1):
            record = parse_log_line(line)

            if record is None:
                invalid_lines.append(line_number)
                continue

            records.append(record)
            level_counts[record["level"]] += 1

    return records, level_counts, invalid_lines


if __name__ == "__main__":
    records, counts, invalid = parse_log_file("application.log")
    print("Parsed records:", len(records))
    print("Levels:", dict(counts))
    print("Invalid lines:", invalid)

The project combines regex with Python Counter, the enumerate function, and safe file handling. For large document workflows, the guide to extracting text from PDFs shows another source of text that may need downstream pattern processing.

Test Regex Patterns

from log_parser import parse_log_line


def test_valid_log_line():
    result = parse_log_line(
        "2026-07-10T12:30:00 INFO Service started"
    )

    assert result == {
        "timestamp": "2026-07-10T12:30:00",
        "level": "INFO",
        "message": "Service started",
    }


def test_invalid_log_line():
    assert parse_log_line("not a log line") is None

Keep examples that should match and should not match. Boundary cases often reveal patterns that are too permissive or too restrictive. The Python unit testing guide covers parameterized test ideas and regression tests.

Performance and Safety

Some ambiguous patterns can cause excessive backtracking on specially constructed input. Risks increase with nested quantifiers and overlapping alternatives, especially when processing untrusted long strings.

• Prefer specific character classes over repeated dots.
• Avoid unnecessary nested quantifiers such as ambiguous repeated groups.
• Limit input length before applying a complex validator.
• Test failure cases, not only successful short examples.
• Use a dedicated parser when the input has a real grammar.
• Consider a simpler two-stage process: locate candidates, then validate with ordinary Python.

A compiled pattern improves reuse but does not repair a poorly designed expression.

When Not to Use Regex

• Use startswith() for a fixed prefix.
• Use in for simple substring membership.
• Use str.replace() for a literal replacement.
• Use split() for one fixed delimiter.
• Use json for JSON.
• Use an HTML or XML parser for markup.
• Use datetime.strptime() to validate an actual calendar date.
• Use URL parsing libraries for URLs rather than one universal URL regex.

The best regex is often the smallest pattern that solves a clearly bounded problem.

Common Mistakes

• Forgetting a raw-string prefix and losing backslashes.
• Calling group() without checking for None.
• Using match() when whole-string validation needs fullmatch().
• Adding capturing groups to findall() and being surprised by tuples or captured-only results.
• Assuming \d means only ASCII digits.
• Using a regex to claim that an email mailbox, date, identifier, or account truly exists.
• Parsing arbitrary HTML with regex.
• Writing one unreadable expression where several simple checks would be safer.

Frequently Asked Questions

Do I need to install re?

No. It is part of the Python standard library.

Why use raw strings?

They prevent Python from interpreting most backslash escapes before the regex engine sees the pattern.

What is the difference between search and fullmatch?

Search looks anywhere in the string. Fullmatch requires the entire string to conform to the pattern.

How do I find overlapping matches?

Ordinary find operations return non-overlapping matches. A lookahead pattern can expose overlapping starting positions, but design and test it carefully.

Should I compile every pattern?

No. Compile patterns that are reused or benefit from a named reusable object. Module functions are fine for one-off expressions.

Can regex validate a date?

It can validate a textual shape. Use the datetime module to validate calendar rules.

Conclusion

Python regex is most effective when the problem is genuinely pattern-based and the accepted text is clearly defined. Use raw strings, choose search() or fullmatch() deliberately, prefer named groups for structured extraction, and test both matching and failing inputs.

Keep patterns small, document complex expressions with verbose mode, and switch to a structured parser when the data has a real format. Those habits make regular expressions useful without turning them into fragile hidden code.