itertools.groupby() groups consecutive elements that produce the same key. It is lazy, consumes input only as needed, and can process large ordered streams without loading everything into memory. Its behavior is different from SQL GROUP BY: equal keys that are not adjacent create separate groups.
This guide covers sorting, shared group iterators, streaming aggregation, object attributes, nested grouping, and common mistakes.
Basic example
from itertools import groupby
values = [1, 1, 2, 2, 2, 3]
for key, group in groupby(values):
print(key, list(group))
Each run of equal consecutive values becomes one group.
Adjacency is the core rule
values = [1, 2, 1]
print([(key, list(group)) for key, group in groupby(values)])
The value 1 appears twice because the matching values are separated. Sort first when all equal keys must be combined.
Sort with the same key
records.sort(key=lambda item: item["category"])
for category, group in groupby(records, key=lambda item: item["category"]):
items = list(group)
process(category, items)
The sorting key and grouping key should represent the same ordering rule.
Groups share the source iterator
Each group is a subiterator connected to the main iterator. Advancing to the next key invalidates the previous group. Materialize a group immediately if it must be retained.
for key, group in groupby(data, key=get_key):
items = list(group)
store(key, items)
Streaming aggregation
for category, group in groupby(records, key=lambda row: row.category):
total = sum(row.amount for row in group)
print(category, total)
The group is consumed directly by sum, keeping memory usage low.
Counting without a list
for key, group in groupby(data, key=get_key):
count = sum(1 for _ in group)
print(key, count)
Grouping objects
from operator import attrgetter
orders.sort(key=attrgetter("customer_id"))
for customer_id, group in groupby(orders, key=attrgetter("customer_id")):
process_orders(customer_id, group)
attrgetter can make a repeated attribute key clearer than multiple lambdas.
Best use case: already ordered input
Database results with ORDER BY, sorted files, and monotonic event streams are excellent inputs. groupby can aggregate them incrementally.
Dictionary versus groupby
A dictionary of lists groups keys regardless of order but retains every item. groupby uses little memory but requires matching keys to be adjacent.
Text normalization
names.sort(key=str.casefold)
for key, group in groupby(names, key=str.casefold):
print(key, list(group))
Use the same normalization for sorting and grouping. Human-language sorting may require locale-aware rules.
Nested grouping
rows.sort(key=lambda row: (row.country, row.city))
for country, country_group in groupby(rows, key=lambda row: row.country):
country_rows = list(country_group)
for city, city_group in groupby(country_rows, key=lambda row: row.city):
process(country, city, city_group)
The outer group is materialized because all subgroups share the underlying iterator.
Common mistakes
- Expecting nonadjacent equal keys to merge.
- Sorting and grouping with incompatible keys.
- Saving a group iterator for later consumption.
- Materializing every group when streaming would work.
- Using a key function with side effects.
Recommended practice
Document ordering requirements, keep key functions pure, consume each group before advancing, and choose materialization only when retention is needed. See the internal guides to Python itertools and itertools.pairwise.
Conclusion
itertools.groupby is ideal for segmenting sorted sequences and building low-memory aggregations. Understanding adjacency and shared iterators is essential for correct results.
