Skip to content

Per-field truncation / summarization for large nested log fields #799

@aress31

Description

@aress31

Summary

It would be useful if structlog provided a built-in processor for per-field truncation / summarization of large values, especially nested dicts and lists.

Problem

Some fields in structured logs are much larger than others, for example:

  • request / response payloads
  • nested kwargs
  • prompt text
  • model outputs

These can overwhelm otherwise readable logs. Global string truncation is not enough; what is often needed is field-aware summarization.

Desired behavior

Something like a built-in processor where selected fields can be limited independently, for example:

  • truncate long strings for prompt
  • limit list length for messages
  • limit depth / keys for nested dicts like payload

Example

log.debug(
    "completion.request",
    prompt=very_large_prompt,
    messages=large_messages,
    extra_body=large_nested_dict,
)

Possible API shape:

structlog.processors.truncate_fields(
    {
        "prompt": {"max_length": 300},
        "messages": {"max_items": 2, "max_string_length": 200},
        "extra_body": {"max_depth": 1, "max_keys": 10},
    }
)

Why

This seems like a common structured-logging need: preserve useful context while keeping summary/debug logs readable without writing a custom processor every time.

Would maintainers be open to something like this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions