Skip to content

monitor_agent: incremental metric collection with windowing and persisted state#56

Merged
seanwevans merged 2 commits into
mainfrom
codex/refactor-metric-query-strategy-in-monitor_agent
Apr 4, 2026
Merged

monitor_agent: incremental metric collection with windowing and persisted state#56
seanwevans merged 2 commits into
mainfrom
codex/refactor-metric-query-strategy-in-monitor_agent

Conversation

@seanwevans

Copy link
Copy Markdown
Owner

Motivation

  • Avoid full-table metric recomputation on each run by filtering queries to a time window or only newly completed rows.
  • Provide a stable, indexable predicate and watermark so recurring CSV outputs append only new intervals.

Description

  • Added CLI window options --since-hours and --since-days and a compute_since_timestamp helper to convert them into a completed_at >= ... filter used in queries.
  • Implemented persisted high-water-mark state via a new monitor_state table and helper functions ensure_monitor_state_table, load_monitor_state, and save_monitor_state to record last_completed_at and last_command_id so the agent can query only rows newer than the watermark.
  • Reworked collect_metrics to accept since_timestamp, last_completed_at, and last_command_id and to build incremental WHERE predicates (completed_at >= %s or (completed_at > %s OR (completed_at = %s AND id > %s))) while preserving streaming server-side cursor behavior and stable ordering ORDER BY day, user_id.
  • Added get_watermark to compute the latest (completed_at, id) after each run and updated CSV mode to append only newly computed intervals because each loop queries only unprocessed rows.
  • Updated schema sql/init_schema.sql to create monitor_state and to add an index commands_status_completed_at_idx ON commands (status, completed_at, id) to support the incremental filter efficiently.
  • Extended tests/test_monitor_agent.py to verify streaming behavior, SQL predicate and bound parameters for incremental mode, stable ordering clause, and window-argument conflict handling.

Testing

  • Ran unit tests for the monitor agent with pytest -q tests/test_monitor_agent.py which reported 4 passed.
  • New tests exercise streaming via the server-side cursor, the incremental WHERE clause and params, CSV flush behavior, and rejection of mutually exclusive window args, and all passed.

Codex Task

@seanwevans
seanwevans merged commit 0112643 into main Apr 4, 2026
2 checks passed
@seanwevans
seanwevans deleted the codex/refactor-metric-query-strategy-in-monitor_agent branch April 4, 2026 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant