Summary
It would be useful if structlog provided a built-in processor for per-field truncation / summarization of large values, especially nested dicts and lists.
Problem
Some fields in structured logs are much larger than others, for example:
- request / response payloads
- nested kwargs
- prompt text
- model outputs
These can overwhelm otherwise readable logs. Global string truncation is not enough; what is often needed is field-aware summarization.
Desired behavior
Something like a built-in processor where selected fields can be limited independently, for example:
- truncate long strings for
prompt
- limit list length for
messages
- limit depth / keys for nested dicts like
payload
Example
log.debug(
"completion.request",
prompt=very_large_prompt,
messages=large_messages,
extra_body=large_nested_dict,
)
Possible API shape:
structlog.processors.truncate_fields(
{
"prompt": {"max_length": 300},
"messages": {"max_items": 2, "max_string_length": 200},
"extra_body": {"max_depth": 1, "max_keys": 10},
}
)
Why
This seems like a common structured-logging need: preserve useful context while keeping summary/debug logs readable without writing a custom processor every time.
Would maintainers be open to something like this?
Summary
It would be useful if
structlogprovided a built-in processor for per-field truncation / summarization of large values, especially nested dicts and lists.Problem
Some fields in structured logs are much larger than others, for example:
These can overwhelm otherwise readable logs. Global string truncation is not enough; what is often needed is field-aware summarization.
Desired behavior
Something like a built-in processor where selected fields can be limited independently, for example:
promptmessagespayloadExample
Possible API shape:
Why
This seems like a common structured-logging need: preserve useful context while keeping summary/debug logs readable without writing a custom processor every time.
Would maintainers be open to something like this?