Skip to content

3b: Binary-safe envelope parser for attachment items #245

Description

@ajianaz

Parent: #233 · Blocked by: #236, #244

Scope

Add binary-safe parsing for attachment envelope items. This is the highest-risk sub-issue in Phase 3 because the existing parser is line-based (text) while attachments contain arbitrary binary data.

Problem

Current parse_envelope_text() reads envelope line-by-line:

{"len":"42"}\n  ← envelope header
{"type":"attachment","length":1024,"filename":"screenshot.png"}\n  ← item header
<1024 bytes of raw binary>\n  ← THIS BREAKS line-based parsing

Line readers split on \n which corrupts binary data containing newline bytes.

Solution: Hybrid parsing strategy

  1. Two-pass approach:

    • Pass 1: Parse envelope header + item headers (all JSON text, line-based)
    • Pass 2: For each attachment item, seek to content_start and read exactly length bytes
  2. Reference implementation sketch:

fn parse_envelope(data: &[u8]) -> Result<ParsedEnvelope, EnvelopeError> {
    // Split into text section (headers) and binary sections (attachments)
    let text_boundary = find_attachment_boundary(data);
    let text_section = &data[..text_boundary];
    
    // Parse text items normally (event, transaction, session)
    let mut result = parse_envelope_text(text_section)?;
    
    // Parse binary attachments from remaining buffer
    let binary_remaining = &data[text_boundary..];
    while !binary_remaining.is_empty() {
        let header = read_header_line(binary_remaining)?;
        if header.item_type == "attachment" {
            let payload = &binary_remaining[header.content_start..header.content_start + header.length];
            result.attachments.push(Attachment {
                filename: header.filename,
                content_type: header.content_type,
                attachment_type: header.attachment_type,
                data: payload.to_vec(),
            });
        }
    }
    Ok(result)
}

Tasks

  • Add find_attachment_boundary() — scan for first item with "type":"attachment"
  • Add read_header_line() — parse JSON header from byte buffer, return offset + length
  • Add parse_envelope() as new entry point (supersedes parse_envelope_text() for binary envelopes)
  • Refactor existing parse_envelope_text() to be called from within parse_envelope() for non-attachment items
  • Test: envelope with 1 attachment + 1 event (both parsed correctly)
  • Test: envelope with binary data containing \n bytes (not corrupted)
  • Test: envelope with NO attachments (backward compatible — identical output to current parser)
  • Test: truncated attachment data (missing bytes) → graceful error, partial parse
  • Benchmark: parser performance on 10KB attachment (must stay under 1ms)

Risks

Risk Mitigation
Binary data with \n breaks line reader Length-based read, not line-based
Large attachments OOM Max attachment size limit (configurable, default 10MB)
Breaking change to parse_envelope_text() Keep old function, add new parse_envelope() as wrapper

Effort: ~3 hours (highest complexity in Phase 3)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestscope:ingestEnvelope ingest / parser changestype:envelopeSentry envelope item type expansion

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions