Useful application logs need context. A message such as “payment failed” becomes much more actionable when it also includes a request identifier, user, service, environment, payment identifier, and processing stage. Python’s logging.LoggerAdapter helps attach shared context to many records. In recent Python versions, the merge_extra parameter makes the adapter more flexible by combining its persistent context with the extra dictionary supplied by an individual logging call.
This guide explains what merge_extra changes, how to use it for structured logging, how collisions are handled, and how to keep sensitive information out of your logs.
What LoggerAdapter does
A LoggerAdapter wraps a normal logger and adds fields to every record. It is useful when a sequence of messages belongs to the same request, background job, customer, or transaction.
import logging
logging.basicConfig(
level=logging.INFO,
format="%(levelname)s %(request_id)s %(message)s",
)
base_logger = logging.getLogger("api")
logger = logging.LoggerAdapter(
base_logger,
{"request_id": "req-123"},
)
logger.info("request received")
The adapter places its dictionary into the resulting LogRecord. A formatter can then include those values in text, JSON, or another output format.
Why merge_extra matters
Without merging, the adapter’s stored context can prevent call-specific fields from being combined in the way developers expect. Teams often worked around this by creating a new adapter for each event, manually copying dictionaries, or subclassing LoggerAdapter.
With merge_extra=True, the persistent fields and the fields passed to a single call are combined. This is ideal when every event needs shared correlation data but also has unique details.
Basic example
import logging
logging.basicConfig(
level=logging.INFO,
format=(
"%(levelname)s request=%(request_id)s "
"user=%(user_id)s order=%(order_id)s %(message)s"
),
)
base_logger = logging.getLogger("checkout")
logger = logging.LoggerAdapter(
base_logger,
{"request_id": "req-987", "user_id": "u-42"},
merge_extra=True,
)
logger.info(
"order validated",
extra={"order_id": "ord-1001"},
)
The record contains all three identifiers. The request and user belong to the adapter, while the order belongs only to this event.
Handling key collisions
When the same key appears in both dictionaries, the call-specific value can replace the adapter value. That can be useful, but it can also weaken the meaning of supposedly stable context.
logger.info(
"action performed on behalf of another user",
extra={"user_id": "u-admin", "order_id": "ord-1002"},
)
If user_id is expected to identify the authenticated user for the entire request, replacing it creates misleading records. Prefer separate names such as authenticated_user_id, actor_user_id, and target_user_id. A logging field dictionary is also an API, so its semantics should be documented.
Per-request logging
A clean pattern is to create one adapter when a request begins and pass it to application services. This avoids global mutable state and keeps dependencies explicit.
def request_logger(base_logger, request_id, user_id):
return logging.LoggerAdapter(
base_logger,
{
"request_id": request_id,
"user_id": user_id,
"service": "payments",
},
merge_extra=True,
)
def process_payment(logger, payment_id):
logger.info(
"payment started",
extra={"payment_id": payment_id, "stage": "start"},
)
For related concepts, see the Academify guides to Python logging, Python contextvars, Python decorators, and Python exceptions.
Structured JSON logs
LoggerAdapter does not produce JSON by itself. It enriches the record so that a JSON formatter can serialize the fields.
class JsonFormatter(logging.Formatter):
def format(self, record):
import json
payload = {
"level": record.levelname,
"message": record.getMessage(),
"request_id": getattr(record, "request_id", None),
"user_id": getattr(record, "user_id", None),
"order_id": getattr(record, "order_id", None),
}
return json.dumps(payload)
Structured records are easier to filter and aggregate in systems such as Elasticsearch, Loki, Datadog, CloudWatch, and OpenSearch. The official Python logging documentation describes the adapter API, while the Logging Cookbook presents advanced patterns.
Avoid reserved LogRecord names
A LogRecord already contains attributes such as name, levelname, filename, module, and message. Attempting to overwrite reserved names through extra can raise an exception. Define a stable naming convention for application fields, such as request_id, customer_id, job_id, operation, and duration_ms.
Protect sensitive data
Never log passwords, access tokens, complete cookies, API keys, payment card data, or unnecessary personal information. Because merging makes it easy to attach arbitrary dictionaries, it also makes accidental leakage easier. Use an allowlist rather than trying to remove known secrets after the fact.
ALLOWED_FIELDS = {"order_id", "payment_id", "stage", "duration_ms"}
def safe_extra(data):
return {
key: value
for key, value in data.items()
if key in ALLOWED_FIELDS
}
Also consider truncating long strings and normalizing values before logging. A malformed or extremely large object should not be allowed to create huge records.
Version compatibility
Confirm the minimum Python version supported by your project before passing merge_extra. Older interpreters may reject the argument. A compatibility helper can inspect the constructor and fall back to the traditional adapter.
import inspect
import logging
def create_adapter(logger, context):
parameters = inspect.signature(logging.LoggerAdapter).parameters
if "merge_extra" in parameters:
return logging.LoggerAdapter(
logger,
context,
merge_extra=True,
)
return logging.LoggerAdapter(logger, context)
If call-specific merging is essential on older versions, implement a small subclass whose process method explicitly combines dictionaries. Test that behavior carefully, especially collision precedence.
Testing enriched records
Tests should verify both the message and the attached fields. In pytest, caplog exposes captured LogRecord instances.
def test_log_context(caplog):
base = logging.getLogger("test")
adapter = logging.LoggerAdapter(
base,
{"request_id": "req-test"},
merge_extra=True,
)
with caplog.at_level(logging.INFO):
adapter.info("ok", extra={"order_id": "ord-test"})
record = caplog.records[0]
assert record.request_id == "req-test"
assert record.order_id == "ord-test"
It is also valuable to test collision behavior, missing optional fields, redaction, and JSON serialization. Logging code often runs during failures, so it should not introduce a second exception that hides the original problem.
Adapters in asynchronous code
Passing an adapter explicitly works well in many async applications. For deeply nested frameworks, however, contextvars may provide a cleaner way to hold request-local values. A logging filter can then copy values from context variables into every record. These techniques are complementary: context variables can supply stable context, while merge_extra adds event-specific fields.
When to use merge_extra
Use it when several records share stable context and individual events need additional structured attributes. Common cases include HTTP requests, queue jobs, command-line operations, data pipelines, scheduled tasks, and distributed transactions.
A plain logger is enough for small scripts. A filter may be better for process-wide fields. A context variable may be better for implicit async context. The adapter is strongest when you want explicit, testable context that travels with a component or operation.
Practical checklist
- Define stable field names and meanings.
- Avoid reserved
LogRecordattributes. - Decide whether call-specific values may override adapter values.
- Use an allowlist for fields derived from user or request data.
- Test formatted output and raw record attributes.
- Confirm Python version compatibility.
- Keep logging failures from masking application errors.
Conclusion
LoggerAdapter with merge_extra is a practical foundation for structured Python logs. It combines persistent correlation fields with event-specific details, reduces repetitive dictionary handling, and keeps context visible in code. With clear naming, collision rules, compatibility checks, tests, and strict protection of sensitive data, it produces logs that are easier to search, debug, and trust.
