[kernel-1116] browser logging: Event Schema & Pipeline

[archandatta]

Note

Medium Risk
Introduces new concurrency-heavy event ingestion and persistence code (ring buffer fan-out, atomic sequencing, and file appends), which can impact reliability and performance under load. Also adds truncation logic that may drop payload data when events exceed the 1MB limit.

Overview
Adds a new events package implementing a canonical BrowserEvent schema (category/source/detail level fields) and a capture pipeline that stamps capture_session_id, monotonic seq, and timestamps, then enforces a 1MB max record size by truncating oversized data and marking events as truncated.

Implements two sinks: a non-blocking in-memory RingBuffer with per-reader cursors and an events.dropped sentinel when readers fall behind, and a per-category JSONL FileWriter that lazily opens <category>.log files and serializes concurrent writes. Includes extensive unit tests covering serialization, overflow/drop behavior, concurrent readers/writers, file routing, sequencing, and truncation.

^{Written by Cursor Bugbot for commit 36cff2d. This will update automatically on new commits. Configure here.}

[rgarcia]

nice direction on the pipeline/schema. one thing i'd consider before cementing BrowserEvent v1: if this is eventually going to drive both capture controls (for example: "turn on cdp console only" or "turn on network request/response capture at headers-only detail") and subscriptions, it might be worth making those selector dimensions first-class in the envelope instead of encoding too much into a single type string.

concretely, i think i'd lean toward:

• keeping the primary event identity semantic, e.g. console.log, network.request, input.click
• adding explicit provenance fields like source_kind (cdp, kernel_api, extension, local_process) plus source_name / source_event (for example Runtime.consoleAPICalled)
• adding an explicit detail_level (minimal, default, verbose, raw)
• possibly making category first-class too instead of deriving it from the type prefix

i probably would not use raw Runtime.* / Network.* as the primary type, since that makes future non-cdp producers feel awkward/second-class. i think the semantic-type + provenance split ages better if we later want to emit events from things like:

• third-party extensions running in the browser and talking to localhost
• vm-local helper processes/programs running alongside the browser
• server/api-driven tool actions like screenshot/input/recording events

that shape also gives the system a much more natural control surface for both capture config and subscriptions, since selectors can operate directly on stable fields like category, topic, source_kind, and detail_level instead of needing to parse overloaded event names.

[Sayan-]

focused review on the pipeline since raf had some feedback to the event schema!

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[cursor]

Marshal error silently writes corrupt JSONL line

Low Severity

truncateIfNeeded silently swallows json.Marshal errors and returns nil data (since Marshal returns nil on error). The caller in Pipeline.Publish passes this nil data to FileWriter.Write, which executes append(nil, '\n') and writes a bare newline to the JSONL file — a corrupt, non-JSON line. The function returns no error, so the pipeline cannot detect the failure. The first error path (return ev, data) and second (return ev, nil) are also inconsistent, suggesting an oversight.

Additional Locations (1)

• server/lib/events/pipeline.go#L50-L55

 

[Sayan-]

==

[Sayan-]

changes look great!

[Sayan-]

==

[Sayan-]

On final review I'm wondering if the drop sentinel should be a separate type from BrowserEvent. A drop notification is stream metadata, not browser content.

A deeper dive a la opus:

The current sentinel has fabricated/zero fields (Seq: 0, Ts: 0, Category: "system", Source: "kernel_api") that aren't actually observed from the browser. Since there are no consumers yet, now's the cheapest time to split this:

type ReadResult struct {
    Event   *BrowserEvent // nil when Dropped > 0
    Dropped uint64        // count of events lost at this point
}

This makes the contract compile-time safe consumers can't accidentally serialize a drop as a real event because Event is nil. The transport layer (e.g. a future WebSocket handler) can decide how to represent drops on the wire independently.

[Sayan-]

might be simpler to just make this an uint64 since we only mutate it while holding the lock. nbd

[Sayan-]

may be helpful to capture the intended use of the pipeline wrt lifecycle (e.g. can you stop and restart the same pipeline, do you need to send a terminal event before closing, etc)