Skip to content

feat: count only records the destination received (BP-1324) - #345

Open
Dylan-M wants to merge 3 commits into
dylanmyers/bp-1302-docs-refreshfrom
dylanmyers/bp-1324-output-delivery-count
Open

Dylan-M wants to merge 3 commits into
dylanmyers/bp-1302-docs-refreshfrom
dylanmyers/bp-1324-output-delivery-count

Conversation

@Dylan-M

@Dylan-M Dylan-M commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Proposed Change

Make generator.count only count a record once the destination has it, such as: a successful socket write, a Splunk ACK, or an OTLP export with nothing rejected. A record that fails or is dropped after it was queued is replaced so that a run ends with exactly the count delivered. Delivery is at least once, so a destination can receive duplicates. docs/finite-generation.md lists the delivery point for each output.

Breaking changes:

  • The end-of-run log field records_emitted is now records_delivered.
  • Validation rejects output.otlp-grpc.batchTimeout: 0 and an output.hec.batchTimeout of 0 or less. With HEC ACKs on, it also rejects an ackPollInterval or ackTimeout of 0 or less. A prometheus-remote-write batchTimeout: 0 is rejected when a count is set.
  • otlp-grpc metric and trace exports now use requestTimeout, as logs do, instead of batchTimeout. Stop no longer cancels an export in flight.

stdout now buffers metrics and spans along with logs. Filegen replay retries a failed line up to three times after the rest of the pass, so retried lines arrive out of order. After the last retry it drops the line with an error log. Permit.Forgo then lowers that cycle's count, so the run still ends.

Checklist
  • Changes are tested
  • CI has passed

@Dylan-M
Dylan-M requested review from a team as code owners October 30, 2026 14:29
@Dylan-M
Dylan-M added this pull request to stack #347 October 30, 2026 14:29
@Dylan-M Dylan-M changed the title dylanmyers/bp 1324 output delivery count feat: count only records the destination received (BP-1324) Oct 30, 2026
@Caleb-Hurshman Caleb-Hurshman self-assigned this Oct 30, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants