diff --git a/README.md b/README.md index 9061a7f..0b02f9b 100644 --- a/README.md +++ b/README.md @@ -54,6 +54,7 @@ Common reports: codex-usage-profiler --days 7 --top 20 codex-usage-profiler --since 2026-06-24 codex-usage-profiler --format json --output reports/latest.json +codex-usage-profiler --days 14 --monthly-plan-price-usd 200 --plan-price Plus=20 --plan-price Pro=200 codex-usage-dashboard --report reports/latest.json ``` @@ -65,9 +66,30 @@ codex-usage-profiler --no-codexbar --paths ~/.codex/sessions Raw prompt and response bodies are not printed by default. Use `--include-snippets` only when you explicitly want bounded redacted task snippets in JSON/text output. +## Network Collection + +For multi-machine setups, install the collector on each host and run the dashboard/report refresh on one always-on machine: + +```bash +codex-usage-collector --print-default-config > ~/.config/codex-usage-profiler/collector.json +codex-usage-collector --config ~/.config/codex-usage-profiler/collector.json --once +``` + +The collector scans configurable JSON/JSONL session globs for Codex, Claude, Pi, T3Chat, Kimi, droid, Cursor, OpenCode, and Paperclip, stages changed files with metadata sidecars, and pushes them to the configured central destination. Linux systemd user units, macOS launchd plist, LXSO report refresh, dashboard service, and a rootless high-port `.lan` proxy are under `ops/`. + +Central LXSO pattern: + +```bash +ops/bin/codex-usage-run-report.sh +codex-usage-dashboard --report ~/.local/state/codex-usage-profiler/latest-report.json --host 0.0.0.0 --port 8775 +``` + +Collector, report-refresh, and dashboard access events are written as JSONL and can also be forwarded to Orange Pi syslog and LXSO2 Loki. In Grafana, query Loki with `{job="codex-usage-profiler"}` and narrow by `service="collector"`, `service="report"`, or `service="dashboard"`. + ## What The Tool Produces - Usage grouped by client, project, task, model, day, hour, Paperclip company, Paperclip staff, and Paperclip task. +- Paperclip company spend by day/month, company 30-day projections, and configurable plan-price comparisons. - Top sessions by observed tokens and rate-card cost. - Current CodexBar quota windows when CodexBar is installed. - CodexBar local-vs-cache ratio rows to reveal scope mismatch. @@ -79,6 +101,7 @@ Raw prompt and response bodies are not printed by default. Use `--include-snippe - `Tokens`: observed tokens found in local session logs. - `Rate-card cost`: directional replacement-cost estimate from known model rates or CodexBar pricing caches. - `Live quota now`: current CodexBar quota window, not a historical usage allocation. +- `Plan comparison`: projected local rate-card replacement cost vs configured plan price; useful for value/scale decisions, not official quota reconstruction. - `Observed share`: share of tokens within the scanned local logs. - `Durable output`: sessions with observable output signals such as edits, tests, commits, PRs, or action tools. This does not prove end-user value. - `Review candidates`: sessions matching patterns worth inspecting. These can overlap with durable-output sessions. @@ -105,6 +128,7 @@ This project was built with OpenSpec as an explicit design trail: - `openspec/changes/build-usage-profiler/`: ingestion, attribution, quota/cost estimates, outcome evidence, reporting. - `openspec/changes/add-session-dashboard-ui/`: compact dashboard, user stories, mockup validation, interaction contract. +- `openspec/changes/add-network-session-collection/`: multi-host collection, LXSO deployment, `.lan` access, and Grafana-visible logging. - `docs/dashboard-final-interaction-contract.md`: card-by-card interaction expectations. - `docs/critical-review.md`: subagent critique summary and which critical fixes were accepted. diff --git a/openspec/changes/add-session-dashboard-ui/design.md b/openspec/changes/add-session-dashboard-ui/design.md index 04f7ac6..412f0fd 100644 --- a/openspec/changes/add-session-dashboard-ui/design.md +++ b/openspec/changes/add-session-dashboard-ui/design.md @@ -58,6 +58,10 @@ Use CSS bars and inline SVG for timeline, heatmap, rankings, coverage, and spend Alternative: Chart.js/D3. Rejected for dependency weight. +### Card-local resets and responsive relayout + +Each filterable visualization exposes a local reset button in the panel header. Active chart items still toggle off directly, but reset buttons make recovery obvious when the active item is clipped, scrolled away, or visually subtle. The Sankey/alluvial flow re-renders on viewport changes because its SVG links and DOM nodes are positioned from the container width. + ### Privacy-safe evidence drawer Show structured evidence such as edits, tests, commits, PRs, commands, token counts, attribution confidence, session IDs, and local paths. Do not show raw prompts unless a report explicitly includes snippets. diff --git a/openspec/changes/add-session-dashboard-ui/specs/dashboard-ui/spec.md b/openspec/changes/add-session-dashboard-ui/specs/dashboard-ui/spec.md index 9eba65f..7585ea4 100644 --- a/openspec/changes/add-session-dashboard-ui/specs/dashboard-ui/spec.md +++ b/openspec/changes/add-session-dashboard-ui/specs/dashboard-ui/spec.md @@ -55,6 +55,12 @@ The dashboard SHALL visualize usage through an hourly timeline, day/hour heatmap - **AND WHEN** the user clicks the active item again - **THEN** the dashboard clears only that item's filter and returns the affected panel toward its default scope without clearing unrelated filters +#### Scenario: Card-local reset +- **WHEN** a visualization panel applies or reflects a local selection such as a flow selection, time brush, company/day selection, heatmap cell, review candidate, coverage bucket, or projection filter +- **THEN** that panel shows an enabled reset control +- **AND WHEN** the user activates the reset control +- **THEN** the dashboard clears that panel's local selection, updates the permalink, and disables the reset control again + ### Requirement: Session Table And Evidence Drawer The dashboard SHALL provide a sortable compact session table and a session evidence drawer showing attribution, token/rate-card cost estimates, outcome, review pattern, edits, tests, command labels, and linked identifiers when available. @@ -76,6 +82,11 @@ The dashboard SHALL provide keyboard-accessible controls, visible focus states, - **WHEN** the user navigates the page with the keyboard - **THEN** interactive controls, table rows, exports, and the evidence drawer are reachable and labelled +#### Scenario: No unintended horizontal overflow +- **WHEN** the dashboard is viewed with both sidebars open at desktop widths or on a narrow mobile viewport +- **THEN** top-level regions, panel headers, heatmap, coverage, and flow panels fit within the viewport +- **AND** only deliberate scroll containers such as the session table may scroll horizontally + ### Requirement: Mockup Validation The implementation SHALL preserve the core structure of the final imagegen mockup: filter rail, KPI strip, main graph grid, session table, and evidence drawer. diff --git a/openspec/changes/add-session-dashboard-ui/tasks.md b/openspec/changes/add-session-dashboard-ui/tasks.md index e6848d8..0c44fce 100644 --- a/openspec/changes/add-session-dashboard-ui/tasks.md +++ b/openspec/changes/add-session-dashboard-ui/tasks.md @@ -24,3 +24,11 @@ - [x] 4.2 Add data-function tests for filtering, sorting, summaries, CSV, and permalink state - [x] 4.3 Run the unit and integration test suite - [x] 4.4 Start the dashboard against the real report and verify local/Tailscale access details + +## 5. Interaction And Layout Hardening + +- [x] 5.1 Add card-local reset controls for all filterable visualization panels +- [x] 5.2 Make company day selections visibly active and reversible +- [x] 5.3 Re-render the Sankey flow after viewport/sidebar size changes +- [x] 5.4 Fix desktop/sidebar and mobile text overflow in panel headers, heatmap, coverage, and topbar +- [x] 5.5 Extend E2E tests for card resets, flow bounds, and viewport overflow diff --git a/openspec/changes/archive/2026-07-04-add-network-session-collection/.openspec.yaml b/openspec/changes/archive/2026-07-04-add-network-session-collection/.openspec.yaml new file mode 100644 index 0000000..43e65ca --- /dev/null +++ b/openspec/changes/archive/2026-07-04-add-network-session-collection/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-07-03 diff --git a/openspec/changes/archive/2026-07-04-add-network-session-collection/design.md b/openspec/changes/archive/2026-07-04-add-network-session-collection/design.md new file mode 100644 index 0000000..ffd2793 --- /dev/null +++ b/openspec/changes/archive/2026-07-04-add-network-session-collection/design.md @@ -0,0 +1,73 @@ +## Context + +The profiler currently scans local paths. Alex's real usage spans several machines and harnesses: this Mac, LXSO1, LXSO2, LXSO3, NurseDroid, Codex, Claude, Pi, T3Chat, Kimi, droid, Cursor, OpenCode, and Paperclip. LXSO1 is the best central app host because it is always on and already runs agent services. `grafana.lan` and Loki now live on LXSO2, while Orange Pi still owns `.lan` DNS/Caddy and the syslog/Postgres observability path. + +## Goals / Non-Goals + +**Goals:** + +- Run Codex Usage Profiler continuously on an LXSO machine. +- Collect session files from all reachable hosts without requiring tool-specific daemons. +- Make new tools easy to add with config globs. +- Preserve source host/tool/path metadata. +- Refresh the central report on a timer and serve the dashboard over LAN/Tailscale. +- Emit collector, report-refresh, and dashboard events to local JSONL logs, Orange Pi syslog/Postgres, and LXSO2 Loki for Grafana. + +**Non-Goals:** + +- Exact billing reconstruction. +- Parsing every proprietary chat database format in the collector. +- Replacing existing Grafana/Loki infrastructure. +- Copying auth files, caches, model downloads, browser profiles, or arbitrary app data. + +## Decisions + +1. Use push collectors over SSH/rsync. + + Rationale: every machine can run a user-level timer and push changed files to LXSO1. No central credentials or inbound collector API are needed. + +2. Use config-driven globs with conservative defaults. + + Rationale: Codex and Claude have known JSONL locations, but T3Chat, Kimi, and droid session paths can vary by install. Defaults catch common session directories; per-host config can add or remove paths. + +3. Stage files under `collector/tool/hash/basename` plus a metadata sidecar. + + Rationale: this avoids path collisions, keeps source metadata, and lets the profiler infer client/host from central paths while source files stay untouched. + +3a. Snapshot lightweight Paperclip topology. + + Rationale: central LXSO reports do not have direct access to the Mac's Paperclip API or local company/agent instruction tree. The collector can safely stage a small `paperclip-metadata.json` snapshot with company, agent, project, and workspace labels. This avoids opaque UUID-only reports while keeping raw prompts and source data out of the snapshot. + +4. Keep the collector dependency-free. + + Rationale: LXSO, NurseDroid, Orange Pi, and macOS all have Python 3. A standard-library collector is easier to install than a daemon with a dependency tree. + +5. Log to JSONL, syslog, and Loki. + + Rationale: JSONL gives local troubleshooting. Syslog matches the existing Orange Pi ingestion path. Direct Loki push makes the same events visible in canonical `grafana.lan` without requiring journald scraping on every host. + +6. Prefer LXSO1 for the dashboard; use a high-port `.lan` proxy if Caddy cannot be modified. + + Rationale: Orange Pi Caddy is root-owned. When passwordless sudo is unavailable, `codex-usage.lan:8775` can still work through wildcard `.lan` DNS plus a user-level proxy. + +## Risks / Trade-offs + +- Tool paths can be wrong or incomplete -> keep config visible and log zero-match tools. +- First collection can copy many files -> skip oversized files and copy only mtime/size changes after the first run. +- Non-Codex logs can lack token fields -> central report labels token-known coverage rather than inventing numbers. +- Remote forwarding can fail independently -> local JSONL logs remain authoritative and forwarding failures do not fail collection. +- `.lan` without a port needs root Caddy changes -> provide a working high-port path and document the root-owned route. + +## Migration Plan + +1. Add collector, tests, and ops assets. +2. Deploy the repo to LXSO1 and install dashboard/report systemd user units. +3. Install collector configs/timers on this Mac, LXSO1, LXSO2, LXSO3, and NurseDroid. +4. Start a user-level Orange Pi LAN proxy if Caddy cannot be patched. +5. Run collectors once, refresh the report, check dashboard health, and query Loki/Grafana labels. +6. Roll back by disabling the user timers/services and deleting the central collection directory. + +## Open Questions + +- Whether to add direct parsers for T3Chat/Kimi/droid browser storage formats after real files are observed. +- Whether LXSO2 Loki should later scrape host journald directly in addition to direct push events. diff --git a/openspec/changes/archive/2026-07-04-add-network-session-collection/proposal.md b/openspec/changes/archive/2026-07-04-add-network-session-collection/proposal.md new file mode 100644 index 0000000..b705894 --- /dev/null +++ b/openspec/changes/archive/2026-07-04-add-network-session-collection/proposal.md @@ -0,0 +1,28 @@ +## Why + +Codex Usage Profiler is useful on one machine, but Alex's agent sessions are spread across the Mac, LXSO hosts, NurseDroid, and tool-specific homes. A network collector lets the profiler answer "what burned quota overnight?" from one LXSO-hosted dashboard instead of requiring ad hoc rsyncs and local-only scans. + +## What Changes + +- Add a portable session collector that scans configurable tool globs for Codex, Claude, Pi, T3Chat, Kimi, droid, Cursor, OpenCode, and Paperclip session files. +- Stage changed files with host/tool metadata and push them to a central LXSO collection directory. +- Add central LXSO report refresh and dashboard service templates. +- Add JSONL/syslog/Loki logging for collectors, report refreshes, and dashboard access events so activity appears in Grafana. +- Add deployment docs and service assets for Linux systemd user timers, macOS launchd, and the home `.lan` access path. + +## Capabilities + +### New Capabilities + +- `network-session-collection`: Collect, centralize, profile, serve, and log multi-host AI coding session files. + +### Modified Capabilities + +None. + +## Impact + +- Adds a standard-library Python collector CLI and tests. +- Adds optional dashboard syslog and Loki forwarding. +- Adds ops files under `ops/` for systemd, launchd, report refresh, and LAN proxy deployment. +- Reads local user-level session files and copies them to the configured central host; it does not mutate source session stores. diff --git a/openspec/changes/archive/2026-07-04-add-network-session-collection/specs/network-session-collection/spec.md b/openspec/changes/archive/2026-07-04-add-network-session-collection/specs/network-session-collection/spec.md new file mode 100644 index 0000000..fc39fa5 --- /dev/null +++ b/openspec/changes/archive/2026-07-04-add-network-session-collection/specs/network-session-collection/spec.md @@ -0,0 +1,76 @@ +## ADDED Requirements + +### Requirement: Configurable Multi-Tool Collection +The system SHALL provide a collector that scans configured session-file globs for known AI coding/chat tools and skips missing paths without failing the run. + +#### Scenario: Missing tool paths +- **WHEN** a host has no Kimi or T3Chat session directory +- **THEN** the collector completes successfully and logs zero matches for those tools + +#### Scenario: Added custom path +- **WHEN** a collector config adds a new glob for a tool +- **THEN** matching files are included in the next collection run + +### Requirement: Incremental Central Push +The system SHALL copy changed session files into a central collection directory with source host, tool, and original path metadata. + +#### Scenario: Changed file copied +- **WHEN** a matched session file has a new size or modification time +- **THEN** the collector stages the file, writes a metadata sidecar, and pushes it to the configured destination + +#### Scenario: Unchanged file skipped +- **WHEN** a matched session file is unchanged from collector state +- **THEN** the collector does not restage that file + +#### Scenario: Paperclip source path attribution +- **WHEN** a collected session comes from a Paperclip company, agent, project, or workspace path +- **THEN** the metadata sidecar preserves the original `source_path` +- **AND** includes structured Paperclip identifiers when they can be parsed from the path + +### Requirement: Paperclip Metadata Snapshot +The system SHALL optionally collect lightweight Paperclip topology metadata so the central report can map IDs to company, staff, project, and workspace names without direct access to the source machine's Paperclip API. + +#### Scenario: Collector snapshots Paperclip topology +- **WHEN** the collector can reach the local Paperclip API or workspace directory +- **THEN** it stages a `paperclip-metadata.json` snapshot containing company IDs/names, issue prefixes, agent IDs/titles, project IDs/names, and workspace hints + +#### Scenario: Central report consumes snapshot +- **WHEN** the central profiler scans collected sessions and metadata snapshots +- **THEN** Paperclip sessions are attributed by explicit metadata, original source path, agent/project indexes, and workspace hints in that precedence order + +### Requirement: LXSO Dashboard Deployment +The system SHALL include service assets that run report refresh and dashboard serving on an LXSO host using the central collection directory. + +#### Scenario: Report refresh +- **WHEN** the report refresh service runs +- **THEN** it scans central collected sessions and writes the latest JSON report atomically + +#### Scenario: Dashboard health +- **WHEN** the dashboard service is running +- **THEN** `/healthz` returns `ok` + +### Requirement: LAN Access +The system SHALL provide a documented `.lan` access path for the dashboard on the home network. + +#### Scenario: Caddy available +- **WHEN** Orange Pi Caddy can be updated +- **THEN** `codex-usage.lan` proxies to the LXSO dashboard service + +#### Scenario: Caddy not writable +- **WHEN** Caddy cannot be updated non-interactively +- **THEN** a user-level high-port `.lan` proxy can expose the dashboard without root access + +### Requirement: Observable Logs +The system SHALL emit structured logs for collector runs, report refreshes, and dashboard HTTP access to local files and optionally to syslog and Loki. + +#### Scenario: Collector log forwarding +- **WHEN** a collector run finishes +- **THEN** a structured event includes host, matched files, staged files, pushed files, skipped files, and errors + +#### Scenario: Dashboard access log forwarding +- **WHEN** the dashboard handles an HTTP request +- **THEN** a structured access event can be forwarded to the configured syslog and Loki endpoints + +#### Scenario: Grafana log visibility +- **WHEN** LXSO2 Grafana has a Loki datasource +- **THEN** collector, report-refresh, and dashboard events can be queried by `job="codex-usage-profiler"` and `service` diff --git a/openspec/changes/archive/2026-07-04-add-network-session-collection/tasks.md b/openspec/changes/archive/2026-07-04-add-network-session-collection/tasks.md new file mode 100644 index 0000000..86f88db --- /dev/null +++ b/openspec/changes/archive/2026-07-04-add-network-session-collection/tasks.md @@ -0,0 +1,38 @@ +## 1. OpenSpec + +- [x] 1.1 Write proposal, design, and network collection spec +- [x] 1.2 Validate the OpenSpec change strictly + +## 2. Collector + +- [x] 2.1 Add a dependency-free collector CLI with default tool globs +- [x] 2.2 Stage files with metadata sidecars and incremental state +- [x] 2.3 Push to local or SSH destinations +- [x] 2.4 Emit local JSONL and optional syslog events + +## 3. Central Services + +- [x] 3.1 Add report refresh and dashboard service assets +- [x] 3.2 Add collector systemd and launchd service assets +- [x] 3.3 Add high-port LAN proxy asset for rootless `.lan` access + +## 4. Tests + +- [x] 4.1 Add collector unit tests +- [x] 4.2 Add dashboard syslog/access test coverage +- [x] 4.3 Run unit, E2E, and OpenSpec validation + +## 5. Deployment + +- [x] 5.1 Deploy central profiler to LXSO1 +- [x] 5.2 Install collectors on reachable machines +- [x] 5.3 Expose a `.lan` URL +- [x] 5.4 Verify report refresh, dashboard health, and Grafana-visible logging + +## 6. Paperclip Attribution Hardening + +- [x] 6.1 Preserve collected `source_path` as first-class session metadata +- [x] 6.2 Add structured Paperclip identifiers to collector sidecars +- [x] 6.3 Add optional Paperclip topology snapshots for central reports +- [x] 6.4 Attribute Paperclip company, project, staff, and task from explicit metadata, source paths, snapshots, and workspace hints +- [x] 6.5 Add regression tests for collected sidecars, prompt preambles, and metadata snapshots diff --git a/openspec/specs/network-session-collection/spec.md b/openspec/specs/network-session-collection/spec.md new file mode 100644 index 0000000..3657c81 --- /dev/null +++ b/openspec/specs/network-session-collection/spec.md @@ -0,0 +1,80 @@ +# network-session-collection Specification + +## Purpose +TBD - created by archiving change add-network-session-collection. Update Purpose after archive. +## Requirements +### Requirement: Configurable Multi-Tool Collection +The system SHALL provide a collector that scans configured session-file globs for known AI coding/chat tools and skips missing paths without failing the run. + +#### Scenario: Missing tool paths +- **WHEN** a host has no Kimi or T3Chat session directory +- **THEN** the collector completes successfully and logs zero matches for those tools + +#### Scenario: Added custom path +- **WHEN** a collector config adds a new glob for a tool +- **THEN** matching files are included in the next collection run + +### Requirement: Incremental Central Push +The system SHALL copy changed session files into a central collection directory with source host, tool, and original path metadata. + +#### Scenario: Changed file copied +- **WHEN** a matched session file has a new size or modification time +- **THEN** the collector stages the file, writes a metadata sidecar, and pushes it to the configured destination + +#### Scenario: Unchanged file skipped +- **WHEN** a matched session file is unchanged from collector state +- **THEN** the collector does not restage that file + +#### Scenario: Paperclip source path attribution +- **WHEN** a collected session comes from a Paperclip company, agent, project, or workspace path +- **THEN** the metadata sidecar preserves the original `source_path` +- **AND** includes structured Paperclip identifiers when they can be parsed from the path + +### Requirement: Paperclip Metadata Snapshot +The system SHALL optionally collect lightweight Paperclip topology metadata so the central report can map IDs to company, staff, project, and workspace names without direct access to the source machine's Paperclip API. + +#### Scenario: Collector snapshots Paperclip topology +- **WHEN** the collector can reach the local Paperclip API or workspace directory +- **THEN** it stages a `paperclip-metadata.json` snapshot containing company IDs/names, issue prefixes, agent IDs/titles, project IDs/names, and workspace hints + +#### Scenario: Central report consumes snapshot +- **WHEN** the central profiler scans collected sessions and metadata snapshots +- **THEN** Paperclip sessions are attributed by explicit metadata, original source path, agent/project indexes, and workspace hints in that precedence order + +### Requirement: LXSO Dashboard Deployment +The system SHALL include service assets that run report refresh and dashboard serving on an LXSO host using the central collection directory. + +#### Scenario: Report refresh +- **WHEN** the report refresh service runs +- **THEN** it scans central collected sessions and writes the latest JSON report atomically + +#### Scenario: Dashboard health +- **WHEN** the dashboard service is running +- **THEN** `/healthz` returns `ok` + +### Requirement: LAN Access +The system SHALL provide a documented `.lan` access path for the dashboard on the home network. + +#### Scenario: Caddy available +- **WHEN** Orange Pi Caddy can be updated +- **THEN** `codex-usage.lan` proxies to the LXSO dashboard service + +#### Scenario: Caddy not writable +- **WHEN** Caddy cannot be updated non-interactively +- **THEN** a user-level high-port `.lan` proxy can expose the dashboard without root access + +### Requirement: Observable Logs +The system SHALL emit structured logs for collector runs, report refreshes, and dashboard HTTP access to local files and optionally to syslog and Loki. + +#### Scenario: Collector log forwarding +- **WHEN** a collector run finishes +- **THEN** a structured event includes host, matched files, staged files, pushed files, skipped files, and errors + +#### Scenario: Dashboard access log forwarding +- **WHEN** the dashboard handles an HTTP request +- **THEN** a structured access event can be forwarded to the configured syslog and Loki endpoints + +#### Scenario: Grafana log visibility +- **WHEN** LXSO2 Grafana has a Loki datasource +- **THEN** collector, report-refresh, and dashboard events can be queried by `job="codex-usage-profiler"` and `service` + diff --git a/ops/bin/codex-usage-run-report.sh b/ops/bin/codex-usage-run-report.sh new file mode 100755 index 0000000..b81e71a --- /dev/null +++ b/ops/bin/codex-usage-run-report.sh @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +set -euo pipefail + +APP_HOME="${CUP_APP_HOME:-$HOME/services/codex-usage-profiler}" +DATA_HOME="${CUP_DATA_HOME:-$HOME/.local/share/codex-usage-profiler}" +STATE_HOME="${CUP_STATE_HOME:-$HOME/.local/state/codex-usage-profiler}" +LOG_HOME="${CUP_LOG_HOME:-$HOME/.local/log/codex-usage-profiler}" +PYTHON="${CUP_PYTHON:-$APP_HOME/.venv/bin/python}" +REPORT="$STATE_HOME/latest-report.json" +TMP_REPORT="$STATE_HOME/latest-report.tmp.json" +LOG_FILE="$LOG_HOME/report-refresh.jsonl" +SYSLOG_HOST="${CUP_SYSLOG_HOST:-192.168.1.30}" +SYSLOG_PORT="${CUP_SYSLOG_PORT:-514}" +LOKI_URL="${CUP_LOKI_URL:-http://192.168.1.221:3100/loki/api/v1/push}" +PORT="${CUP_PORT:-8775}" + +mkdir -p "$STATE_HOME" "$LOG_HOME" "$DATA_HOME/collected-sessions" +exec 9>"$STATE_HOME/report-refresh.lock" +if ! flock -n 9; then + echo '{"event":"report_refresh_skipped","reason":"already_running"}' + exit 0 +fi +export CODEXBAR_COLLECTED_ROOT="$DATA_HOME/collected-sessions" + +paths=("$DATA_HOME/collected-sessions") + +started="$(date +%s)" +"$PYTHON" -m codex_usage_profiler \ + --codexbar-timeout 10 \ + --paths "${paths[@]}" \ + --format json \ + --output "$TMP_REPORT" \ + --monthly-plan-price-usd "${CUP_MONTHLY_PLAN_PRICE_USD:-200}" \ + --plan-price Plus=20 \ + --plan-price Pro=200 +mv "$TMP_REPORT" "$REPORT" + +"$PYTHON" - "$REPORT" "$LOG_FILE" "$SYSLOG_HOST" "$SYSLOG_PORT" "$LOKI_URL" "$started" "$PORT" <<'PY' +import json +import sys +import time +from pathlib import Path + +from codex_usage_profiler.syslog_util import emit_loki_json, emit_syslog_json + +report_path = Path(sys.argv[1]) +log_path = Path(sys.argv[2]) +syslog_host = sys.argv[3] +syslog_port = int(sys.argv[4]) +loki_url = sys.argv[5] +started = int(sys.argv[6]) +port = sys.argv[7] +payload = json.loads(report_path.read_text(encoding="utf-8")) +tokens = sum((s.get("usage") or {}).get("total_tokens", 0) for s in payload.get("sessions", [])) +cost = sum((s.get("estimate") or {}).get("cost_usd") or 0.0 for s in payload.get("sessions", [])) +event = { + "event": "report_refresh", + "sessions": len(payload.get("sessions", [])), + "tokens": tokens, + "cost_usd": round(cost, 6), + "report_path": str(report_path), + "dashboard_port": port, + "duration_seconds": round(time.time() - started, 3), + "warnings": payload.get("warnings", [])[:10], +} +log_path.parent.mkdir(parents=True, exist_ok=True) +with log_path.open("a", encoding="utf-8") as fh: + fh.write(json.dumps(event, sort_keys=True) + "\n") +emit_syslog_json(event, host=syslog_host, port=syslog_port, tag="codex-usage-profiler") +emit_loki_json(event, url=loki_url, labels={"service": "report", "host": "lxso1"}) +print(json.dumps(event, sort_keys=True)) +PY diff --git a/ops/bin/tcp_proxy.py b/ops/bin/tcp_proxy.py new file mode 100755 index 0000000..01d335f --- /dev/null +++ b/ops/bin/tcp_proxy.py @@ -0,0 +1,49 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import selectors +import socket + + +def main() -> int: + parser = argparse.ArgumentParser(description="Tiny TCP forwarder for rootless .lan access.") + parser.add_argument("--listen-host", default="0.0.0.0") + parser.add_argument("--listen-port", type=int, required=True) + parser.add_argument("--target-host", required=True) + parser.add_argument("--target-port", type=int, required=True) + args = parser.parse_args() + + server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + server.bind((args.listen_host, args.listen_port)) + server.listen(50) + while True: + client, _ = server.accept() + target = socket.create_connection((args.target_host, args.target_port), timeout=10) + _bridge(client, target) + + +def _bridge(left: socket.socket, right: socket.socket) -> None: + sel = selectors.DefaultSelector() + left.setblocking(False) + right.setblocking(False) + sel.register(left, selectors.EVENT_READ, right) + sel.register(right, selectors.EVENT_READ, left) + try: + while True: + for key, _ in sel.select(): + src = key.fileobj + dst = key.data + data = src.recv(65536) + if not data: + return + dst.sendall(data) + finally: + sel.close() + left.close() + right.close() + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/ops/config/collector.example.json b/ops/config/collector.example.json new file mode 100644 index 0000000..14fc0d1 --- /dev/null +++ b/ops/config/collector.example.json @@ -0,0 +1,21 @@ +{ + "collector_name": "HOSTNAME", + "destination": "saphid@lxso1:/home/saphid/.local/share/codex-usage-profiler/collected-sessions", + "include_defaults": true, + "syslog_host": "192.168.1.30", + "syslog_port": 514, + "syslog_protocol": "tcp", + "loki_url": "http://192.168.1.221:3100/loki/api/v1/push", + "tools": { + "t3chat": [ + "~/.t3chat/**/*.jsonl", + "~/.config/t3chat/**/*.jsonl" + ], + "kimi": [ + "~/.kimi/**/*.jsonl" + ], + "droid": [ + "~/.droid/**/*.jsonl" + ] + } +} diff --git a/ops/grafana/loki-datasource.yml b/ops/grafana/loki-datasource.yml new file mode 100644 index 0000000..889b930 --- /dev/null +++ b/ops/grafana/loki-datasource.yml @@ -0,0 +1,7 @@ +apiVersion: 1 +datasources: + - name: LXSO2-Loki + type: loki + access: proxy + url: http://loki:3100 + editable: true diff --git a/ops/launchd/com.alxs.codex-usage-collector.plist b/ops/launchd/com.alxs.codex-usage-collector.plist new file mode 100644 index 0000000..26c8d14 --- /dev/null +++ b/ops/launchd/com.alxs.codex-usage-collector.plist @@ -0,0 +1,23 @@ + + + + + Label + com.alxs.codex-usage-collector + ProgramArguments + + /Users/saphid/Library/Application Support/PA-Agent/codex-usage-profiler-venv/bin/codex-usage-collector + --config + /Users/saphid/.config/codex-usage-profiler/collector.json + --once + + StartInterval + 300 + RunAtLoad + + StandardOutPath + /Users/saphid/Library/Logs/codex-usage-collector.launchd.log + StandardErrorPath + /Users/saphid/Library/Logs/codex-usage-collector.launchd.log + + diff --git a/ops/systemd/codex-usage-collector.service b/ops/systemd/codex-usage-collector.service new file mode 100644 index 0000000..886db08 --- /dev/null +++ b/ops/systemd/codex-usage-collector.service @@ -0,0 +1,8 @@ +[Unit] +Description=Collect local AI session files for Codex Usage Profiler +After=network-online.target + +[Service] +Type=oneshot +WorkingDirectory=%h/services/codex-usage-profiler +ExecStart=%h/services/codex-usage-profiler/.venv/bin/codex-usage-collector --config %h/.config/codex-usage-profiler/collector.json --once diff --git a/ops/systemd/codex-usage-collector.timer b/ops/systemd/codex-usage-collector.timer new file mode 100644 index 0000000..22e89a8 --- /dev/null +++ b/ops/systemd/codex-usage-collector.timer @@ -0,0 +1,11 @@ +[Unit] +Description=Collect local AI session files for Codex Usage Profiler every five minutes + +[Timer] +OnBootSec=90s +OnUnitActiveSec=5min +RandomizedDelaySec=60s +Persistent=true + +[Install] +WantedBy=timers.target diff --git a/ops/systemd/codex-usage-dashboard.service b/ops/systemd/codex-usage-dashboard.service new file mode 100644 index 0000000..5c921bc --- /dev/null +++ b/ops/systemd/codex-usage-dashboard.service @@ -0,0 +1,18 @@ +[Unit] +Description=Codex Usage Profiler dashboard +After=network-online.target + +[Service] +Type=simple +Environment=CUP_APP_HOME=%h/services/codex-usage-profiler +Environment=CUP_STATE_HOME=%h/.local/state/codex-usage-profiler +Environment=CUP_SYSLOG_HOST=192.168.1.30 +Environment=CUP_LOKI_URL=http://192.168.1.221:3100/loki/api/v1/push +Environment=CUP_PORT=8775 +WorkingDirectory=%h/services/codex-usage-profiler +ExecStart=%h/services/codex-usage-profiler/.venv/bin/codex-usage-dashboard --report %h/.local/state/codex-usage-profiler/latest-report.json --host 0.0.0.0 --port ${CUP_PORT} --syslog-host ${CUP_SYSLOG_HOST} --loki-url ${CUP_LOKI_URL} +Restart=on-failure +RestartSec=5 + +[Install] +WantedBy=default.target diff --git a/ops/systemd/codex-usage-lan-proxy.service b/ops/systemd/codex-usage-lan-proxy.service new file mode 100644 index 0000000..c99d1ac --- /dev/null +++ b/ops/systemd/codex-usage-lan-proxy.service @@ -0,0 +1,13 @@ +[Unit] +Description=Rootless codex-usage.lan high-port proxy to LXSO1 +After=network-online.target + +[Service] +Type=simple +WorkingDirectory=%h/services/codex-usage-profiler +ExecStart=/usr/bin/env python3 %h/services/codex-usage-profiler/ops/bin/tcp_proxy.py --listen-host 0.0.0.0 --listen-port 8775 --target-host 192.168.1.109 --target-port 8775 +Restart=on-failure +RestartSec=5 + +[Install] +WantedBy=default.target diff --git a/ops/systemd/codex-usage-report.service b/ops/systemd/codex-usage-report.service new file mode 100644 index 0000000..ceaa94c --- /dev/null +++ b/ops/systemd/codex-usage-report.service @@ -0,0 +1,10 @@ +[Unit] +Description=Refresh Codex Usage Profiler report +After=network-online.target + +[Service] +Type=oneshot +Environment=CUP_APP_HOME=%h/services/codex-usage-profiler +Environment=CUP_SYSLOG_HOST=192.168.1.30 +WorkingDirectory=%h/services/codex-usage-profiler +ExecStart=%h/services/codex-usage-profiler/ops/bin/codex-usage-run-report.sh diff --git a/ops/systemd/codex-usage-report.timer b/ops/systemd/codex-usage-report.timer new file mode 100644 index 0000000..e89f6b2 --- /dev/null +++ b/ops/systemd/codex-usage-report.timer @@ -0,0 +1,11 @@ +[Unit] +Description=Refresh Codex Usage Profiler report every fifteen minutes + +[Timer] +OnBootSec=2min +OnUnitActiveSec=15min +RandomizedDelaySec=45s +Persistent=true + +[Install] +WantedBy=timers.target diff --git a/pyproject.toml b/pyproject.toml index de149ee..e6b7d7c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -17,6 +17,8 @@ codex-usage-profiler = "codex_usage_profiler.cli:main" cup = "codex_usage_profiler.cli:main" codex-usage-dashboard = "codex_usage_profiler.dashboard:main" cup-dashboard = "codex_usage_profiler.dashboard:main" +codex-usage-collector = "codex_usage_profiler.collector:main" +cup-collector = "codex_usage_profiler.collector:main" [tool.setuptools.packages.find] where = ["src"] diff --git a/src/codex_usage_profiler/analysis.py b/src/codex_usage_profiler/analysis.py index 4cb8df0..2e99ca9 100644 --- a/src/codex_usage_profiler/analysis.py +++ b/src/codex_usage_profiler/analysis.py @@ -1,10 +1,11 @@ from __future__ import annotations from collections import Counter, defaultdict -from typing import Dict, Iterable, List, Optional, Tuple +from typing import Any, Dict, Iterable, List, Optional, Tuple +from .config import Config from .models import Attribution, Finding, SessionRecord -from .util import day_key, hour_key +from .util import day_key, hour_key, iso_to_datetime def classify_outcomes(records: List[SessionRecord]) -> None: @@ -104,6 +105,53 @@ def build_aggregates(records: List[SessionRecord]) -> Dict[str, List[Dict[str, o } +def build_paperclip_spend(records: List[SessionRecord], config: Config) -> Dict[str, Any]: + paperclip_records = [record for record in records if record.paperclip_company.label != "unknown"] + daily = _period_rows(paperclip_records, "day") + monthly = _period_rows(paperclip_records, "month") + totals = _company_totals(paperclip_records, config.projection_days) + return { + "projection_days": config.projection_days, + "attributed_sessions": len(paperclip_records), + "attributed_tokens": sum(record.usage.get("total_tokens", 0) for record in paperclip_records), + "attributed_cost_usd": sum(record.estimate.cost_usd or 0.0 for record in paperclip_records), + "company_totals": totals, + "daily": daily, + "monthly": monthly, + } + + +def build_plan_analysis(records: List[SessionRecord], config: Config) -> Dict[str, Any]: + span_days = _observed_span_days(records) + observed_cost = sum(record.estimate.cost_usd or 0.0 for record in records) + projected = observed_cost / span_days * config.projection_days if span_days else 0.0 + plans = dict(config.plan_prices_usd) + if config.monthly_plan_price_usd is not None: + plans.setdefault("configured_monthly_plan", config.monthly_plan_price_usd) + rows = [] + for name, price in sorted(plans.items(), key=lambda item: item[1]): + ratio = projected / price if price else None + rows.append( + { + "plan": name, + "monthly_price_usd": price, + "projected_rate_card_cost_usd": projected, + "projected_vs_price_percent": ratio * 100 if ratio is not None else None, + "delta_usd": projected - price, + "note": "replacement_cost_above_plan_price" if projected > price else "replacement_cost_below_plan_price", + } + ) + return { + "projection_days": config.projection_days, + "observed_span_days": span_days, + "observed_cost_usd": observed_cost, + "projected_cost_usd": projected, + "monthly_plan_price_usd": config.monthly_plan_price_usd, + "plans": rows, + "confidence_note": "Projected from local observed rate-card replacement cost. This compares value, not official subscription quota or invoice spend.", + } + + def reconcile_codexbar(records: List[SessionRecord], cost_usage: Optional[Dict[str, object]]) -> List[Dict[str, object]]: if not cost_usage: return [] @@ -153,6 +201,128 @@ def reconcile_codexbar(records: List[SessionRecord], cost_usage: Optional[Dict[s return sorted(rows, key=lambda row: abs((row.get("codexbar_tokens") or 0) - (row.get("local_tokens") or 0)), reverse=True) +def _period_rows(records: List[SessionRecord], period: str) -> List[Dict[str, Any]]: + buckets: Dict[Tuple[str, str], Dict[str, Any]] = {} + for record in records: + key = _period_key(record.start_time, period) + company = record.paperclip_company.label + bucket = buckets.setdefault( + (key, company), + { + "period": key, + "company": company, + "sessions": 0, + "input_tokens": 0, + "cached_input_tokens": 0, + "output_tokens": 0, + "total_tokens": 0, + "estimated_cost_usd": 0.0, + "estimated_credits": 0.0, + "top_staff": Counter(), + "top_projects": Counter(), + "models": Counter(), + "outcomes": Counter(), + }, + ) + _add_record_to_bucket(bucket, record) + return [_finalize_counter_bucket(bucket) for bucket in sorted(buckets.values(), key=lambda row: (str(row["period"]), -float(row["estimated_cost_usd"]), str(row["company"])))] + + +def _company_totals(records: List[SessionRecord], projection_days: int) -> List[Dict[str, Any]]: + buckets: Dict[str, Dict[str, Any]] = {} + dates_by_company: Dict[str, set[str]] = defaultdict(set) + for record in records: + company = record.paperclip_company.label + bucket = buckets.setdefault( + company, + { + "company": company, + "sessions": 0, + "input_tokens": 0, + "cached_input_tokens": 0, + "output_tokens": 0, + "total_tokens": 0, + "estimated_cost_usd": 0.0, + "estimated_credits": 0.0, + "top_staff": Counter(), + "top_projects": Counter(), + "models": Counter(), + "outcomes": Counter(), + "first_day": "unknown", + "last_day": "unknown", + "active_days": 0, + "observed_span_days": 1, + "avg_daily_cost_usd": 0.0, + "active_day_avg_cost_usd": 0.0, + "projected_cost_usd": 0.0, + }, + ) + _add_record_to_bucket(bucket, record) + day = day_key(record.start_time) + if day != "unknown": + dates_by_company[company].add(day) + rows: List[Dict[str, Any]] = [] + for company, bucket in buckets.items(): + dates = sorted(dates_by_company.get(company, set())) + active_days = len(dates) + span_days = _date_span_days(dates) if dates else 1 + cost = float(bucket["estimated_cost_usd"]) + bucket["first_day"] = dates[0] if dates else "unknown" + bucket["last_day"] = dates[-1] if dates else "unknown" + bucket["active_days"] = active_days + bucket["observed_span_days"] = span_days + bucket["avg_daily_cost_usd"] = cost / span_days if span_days else 0.0 + bucket["active_day_avg_cost_usd"] = cost / active_days if active_days else 0.0 + bucket["projected_cost_usd"] = (cost / span_days * projection_days) if span_days else 0.0 + rows.append(_finalize_counter_bucket(bucket)) + return sorted(rows, key=lambda row: float(row["estimated_cost_usd"]), reverse=True) + + +def _add_record_to_bucket(bucket: Dict[str, Any], record: SessionRecord) -> None: + bucket["sessions"] = int(bucket["sessions"]) + 1 + for token_key in ["input_tokens", "cached_input_tokens", "output_tokens", "total_tokens"]: + bucket[token_key] = int(bucket[token_key]) + record.usage.get(token_key, 0) + bucket["estimated_cost_usd"] = float(bucket["estimated_cost_usd"]) + (record.estimate.cost_usd or 0.0) + bucket["estimated_credits"] = float(bucket["estimated_credits"]) + (record.estimate.credits or 0.0) + bucket["top_staff"][record.paperclip_staff.label] += 1 + bucket["top_projects"][record.paperclip_project.label] += 1 + bucket["models"][record.model or "unknown"] += 1 + bucket["outcomes"][record.outcome.label] += 1 + + +def _finalize_counter_bucket(bucket: Dict[str, Any]) -> Dict[str, Any]: + result = dict(bucket) + for key in ["top_staff", "top_projects", "models", "outcomes"]: + value = result.get(key) + if isinstance(value, Counter): + result[key] = dict(value.most_common(5)) + return result + + +def _period_key(value: Optional[str], period: str) -> str: + parsed = iso_to_datetime(value) + if not parsed: + return "unknown" + if period == "month": + return parsed.strftime("%Y-%m") + return parsed.date().isoformat() + + +def _observed_span_days(records: List[SessionRecord]) -> int: + dates = sorted({day_key(record.start_time) for record in records if day_key(record.start_time) != "unknown"}) + return _date_span_days(dates) if dates else 1 + + +def _date_span_days(dates: List[str]) -> int: + if not dates: + return 1 + first = iso_to_datetime(dates[0] + "T00:00:00") + last = iso_to_datetime(dates[-1] + "T00:00:00") + if not first or not last: + return 1 + return max(1, (last.date() - first.date()).days + 1) + + def _key(record: SessionRecord, key: str) -> str: if key == "client": return record.client.label diff --git a/src/codex_usage_profiler/attribution.py b/src/codex_usage_profiler/attribution.py index 9845713..0447120 100644 --- a/src/codex_usage_profiler/attribution.py +++ b/src/codex_usage_profiler/attribution.py @@ -18,7 +18,7 @@ def attribute_sessions(records: list[SessionRecord], config: Config) -> None: def attribute_client(record: SessionRecord, config: Config) -> Attribution: haystack = " ".join( value or "" - for value in [record.originator, record.source, record.thread_source, record.cwd, record.path] + for value in [record.originator, record.source, record.thread_source, record.cwd, record.path, record.source_path] ).lower() for pattern, label in config.client_aliases.items(): if pattern.lower() in haystack: @@ -31,13 +31,23 @@ def attribute_client(record: SessionRecord, config: Config) -> Attribution: return Attribution("Codex exec", "medium", ["originator/source=codex_exec"]) if "cursor" in haystack: return Attribution("Cursor", "medium", ["cursor signal"]) + if "/claude/" in haystack or "/.claude/" in haystack or "claude" in haystack: + return Attribution("Claude Code", "medium", ["claude path/signal"]) if "/.pi/" in haystack or "pi coding" in haystack: return Attribution("Pi Coding Agent", "medium", ["pi path/signal"]) + if "/t3chat/" in haystack or "t3chat" in haystack or "t3 chat" in haystack: + return Attribution("T3Chat", "medium", ["t3chat path/signal"]) + if "/kimi/" in haystack or "kimi" in haystack: + return Attribution("Kimi", "medium", ["kimi path/signal"]) + if "/droid/" in haystack or "droid" in haystack: + return Attribution("droid", "medium", ["droid path/signal"]) + if "/opencode/" in haystack or "opencode" in haystack or "open-code" in haystack: + return Attribution("OpenCode", "medium", ["opencode path/signal"]) return Attribution("unknown", "low", ["no known client signal"]) def attribute_project(record: SessionRecord, config: Config) -> Attribution: - candidates = [record.cwd] + record.workspace_roots + [record.path] + candidates = [record.cwd] + record.workspace_roots + [record.source_path, record.path] for value in candidates: if not value: continue @@ -61,6 +71,9 @@ def _project_from_path(value: str) -> Optional[str]: match = re.search(r"/projects/([^/]+)", expanded) if match: return match.group(1) + match = re.search(r"/collected-sessions/([^/]+)/([^/]+)/", expanded) + if match: + return f"{match.group(1)}:{match.group(2)}" path = Path(expanded) if path.name and path.name not in {".codex", "sessions", "archived_sessions"}: return path.name @@ -83,4 +96,3 @@ def attribute_task(record: SessionRecord, config: Config) -> Attribution: if record.path: return Attribution(f"session:{Path(record.path).stem}", "low", ["session filename"]) return Attribution("unknown", "low", ["no task signal"]) - diff --git a/src/codex_usage_profiler/cli.py b/src/codex_usage_profiler/cli.py index 5374925..619451b 100644 --- a/src/codex_usage_profiler/cli.py +++ b/src/codex_usage_profiler/cli.py @@ -6,7 +6,14 @@ from pathlib import Path from typing import Optional -from .analysis import build_aggregates, classify_outcomes, find_waste_candidates, reconcile_codexbar +from .analysis import ( + build_aggregates, + build_paperclip_spend, + build_plan_analysis, + classify_outcomes, + find_waste_candidates, + reconcile_codexbar, +) from .attribution import attribute_sessions from .codexbar import collect_codexbar from .config import load_config @@ -31,6 +38,9 @@ def main(argv: Optional[list[str]] = None) -> int: parser.add_argument("--no-codexbar", action="store_true", help="Disable CodexBar CLI/cache import") parser.add_argument("--codexbar-timeout", type=int, default=30) parser.add_argument("--include-snippets", action="store_true", help="Include bounded redacted first-request snippets") + parser.add_argument("--monthly-plan-price-usd", type=float, help="Compare projected replacement cost to this monthly plan price") + parser.add_argument("--plan-price", action="append", default=[], metavar="NAME=USD", help="Add a named monthly plan price comparison") + parser.add_argument("--projection-days", type=int, help="Projection horizon for plan/company spend, default 30") parser.add_argument("--report", default="all", choices=["all", "sessions", "summary"], help="Reserved report selector") args = parser.parse_args(argv) @@ -40,8 +50,20 @@ def main(argv: Optional[list[str]] = None) -> int: since = since_date.isoformat() config = load_config(args.config) + if args.monthly_plan_price_usd is not None: + config.monthly_plan_price_usd = args.monthly_plan_price_usd + if args.projection_days is not None: + config.projection_days = max(1, args.projection_days) + for item in args.plan_price: + name, sep, price = item.partition("=") + if not sep: + parser.error("--plan-price must be NAME=USD") + try: + config.plan_prices_usd[name] = float(price) + except ValueError: + parser.error("--plan-price price must be numeric") telemetry = collect_codexbar(enabled=(not args.no_codexbar and config.codexbar_enabled), timeout=args.codexbar_timeout) - paperclip_index = build_paperclip_index(config) + paperclip_index = build_paperclip_index(config, args.paths) records, warnings = parse_logs(args.paths, since=since) attribute_sessions(records, config) apply_paperclip_attribution(records, paperclip_index) @@ -49,13 +71,15 @@ def main(argv: Optional[list[str]] = None) -> int: apply_estimates(records, config, telemetry) classify_outcomes(records) aggregates = build_aggregates(records) + paperclip_spend = build_paperclip_spend(records, config) + plan_analysis = build_plan_analysis(records, config) findings = find_waste_candidates(records) reconciliation = reconcile_codexbar(records, telemetry.cost_usage) if args.format == "json": - output = render_json(records, aggregates, findings, telemetry, reconciliation, warnings, args.include_snippets) + output = render_json(records, aggregates, findings, telemetry, reconciliation, warnings, paperclip_spend, plan_analysis, args.include_snippets) else: - output = render_text(records, aggregates, findings, telemetry, reconciliation, warnings, args.top, args.include_snippets) + output = render_text(records, aggregates, findings, telemetry, reconciliation, warnings, paperclip_spend, plan_analysis, args.top, args.include_snippets) if args.output: path = Path(args.output).expanduser() diff --git a/src/codex_usage_profiler/codexbar.py b/src/codex_usage_profiler/codexbar.py index e409ec0..9d066ec 100644 --- a/src/codex_usage_profiler/codexbar.py +++ b/src/codex_usage_profiler/codexbar.py @@ -1,6 +1,7 @@ from __future__ import annotations import json +import os import plistlib import shutil import subprocess @@ -31,6 +32,7 @@ def collect_codexbar(enabled: bool = True, timeout: int = 30) -> CodexBarTelemet telemetry.history_path = _existing_path(CODEXBAR_HISTORY) telemetry.cost_cache_paths = _glob_existing(CODEXBAR_COST_DIR, "*.json") telemetry.pricing_cache_paths = _glob_existing(CODEXBAR_PRICING_DIR, "*.json") + _merge_collected_codexbar(telemetry, os.environ.get("CODEXBAR_COLLECTED_ROOT")) telemetry.available = bool(telemetry.cli_path or telemetry.cost_cache_paths or telemetry.history_path) if telemetry.cli_path: @@ -64,6 +66,37 @@ def _glob_existing(root: str, pattern: str) -> List[str]: return [str(path) for path in sorted(expanded.glob(pattern), key=lambda p: p.stat().st_mtime, reverse=True)] +def _merge_collected_codexbar(telemetry: CodexBarTelemetry, root: Optional[str]) -> None: + if not root: + return + expanded = Path(root).expanduser() + if not expanded.exists(): + return + paths = [path for path in expanded.rglob("*.json") if "/codexbar/" in str(path)] + histories = sorted( + [path for path in paths if path.name == "codex.json"], + key=lambda p: p.stat().st_mtime, + reverse=True, + ) + if histories and not telemetry.history_path: + telemetry.history_path = str(histories[0]) + cost_paths = [str(path) for path in paths if path.name.startswith(("codex-v", "pi-sessions-v"))] + pricing_paths = [str(path) for path in paths if path.name.startswith("models-")] + telemetry.cost_cache_paths = _dedupe_paths(telemetry.cost_cache_paths + sorted(cost_paths, reverse=True)) + telemetry.pricing_cache_paths = _dedupe_paths(telemetry.pricing_cache_paths + sorted(pricing_paths, reverse=True)) + + +def _dedupe_paths(paths: List[str]) -> List[str]: + seen = set() + result = [] + for path in paths: + if path in seen: + continue + seen.add(path) + result.append(path) + return result + + def _app_version() -> Optional[str]: info = CODEXBAR_APP / "Contents" / "Info.plist" if not info.exists(): @@ -326,4 +359,3 @@ def _int(value: Any) -> int: def _float(value: Any) -> Optional[float]: return float(value) if isinstance(value, (int, float)) else None - diff --git a/src/codex_usage_profiler/collector.py b/src/codex_usage_profiler/collector.py new file mode 100644 index 0000000..4a063aa --- /dev/null +++ b/src/codex_usage_profiler/collector.py @@ -0,0 +1,583 @@ +from __future__ import annotations + +import argparse +import fnmatch +import glob +import hashlib +import json +import os +import re +import shutil +import socket +import subprocess +import sys +import time +import urllib.error +import urllib.request +from pathlib import Path +from typing import Any, Dict, Iterable, List, Optional, Tuple + +from .syslog_util import emit_loki_json, emit_syslog_json + + +DEFAULT_TOOL_GLOBS: Dict[str, List[str]] = { + "codex": [ + "~/.codex/sessions/**/*.jsonl", + "~/.codex/archived_sessions/**/*.jsonl", + "~/.paperclip/instances/default/companies/*/codex-home/sessions/**/*.jsonl", + "~/.paperclip/instances/default/companies/*/codex-home/archived_sessions/**/*.jsonl", + "~/.paperclip/instances/default/companies/*/agents/*/codex-home/sessions/**/*.jsonl", + "~/.paperclip/instances/default/companies/*/agents/*/codex-home/archived_sessions/**/*.jsonl", + ], + "claude": [ + "~/.claude/projects/**/*.jsonl", + "~/.claude/sessions/**/*.jsonl", + "~/.claude/**/sessions/**/*.jsonl", + ], + "pi": [ + "~/.pi/agent/sessions/**/*.jsonl", + "~/.pi/agent/**/sessions/**/*.jsonl", + "~/.pi/agent/projects/**/*.jsonl", + "~/.pi/agent/history/**/*.jsonl", + "~/.pi/agent/runs/**/*.jsonl", + ], + "t3chat": [ + "~/.t3chat/**/*.jsonl", + "~/.config/t3chat/**/*.jsonl", + "~/.local/share/t3chat/**/*.jsonl", + "~/Library/Application Support/T3 Chat/**/*.jsonl", + ], + "kimi": [ + "~/.kimi/**/*.jsonl", + "~/.config/kimi/**/*.jsonl", + "~/.local/share/kimi/**/*.jsonl", + ], + "droid": [ + "~/.droid/**/*.jsonl", + "~/.config/droid/**/*.jsonl", + "~/.local/share/droid/**/*.jsonl", + "~/.nursedroid/**/*.jsonl", + ], + "cursor": [ + "~/.cursor/**/*.jsonl", + "~/Library/Application Support/Cursor/User/workspaceStorage/**/*.jsonl", + ], + "opencode": [ + "~/.opencode/**/*.jsonl", + "~/.local/share/opencode/**/*.jsonl", + ], + "codexbar": [ + "~/Library/Application Support/com.steipete.codexbar/history/codex.json", + "~/Library/Caches/CodexBar/cost-usage/*.json", + "~/Library/Caches/CodexBar/model-pricing/*.json", + "~/Library/Group Containers/Y5PE65HELJ.com.steipete.codexbar/widget-snapshot.json", + ], +} + +DEFAULT_CONFIG = { + "collector_name": None, + "destination": "saphid@lxso1:/home/saphid/.local/share/codex-usage-profiler/collected-sessions", + "include_defaults": True, + "tools": {}, + "exclude": [ + "**/node_modules/**", + "**/.venv/**", + "**/venv/**", + "**/auth.json", + "**/credentials.json", + "**/models_cache.json", + "**/cache/**", + "**/Caches/**", + ], + "max_file_bytes": 104857600, + "state_path": "~/.local/state/codex-usage-collector/state.json", + "staging_dir": "~/.local/share/codex-usage-collector/staging", + "log_path": "~/.local/log/codex-usage-collector/collector.jsonl", + "syslog_host": "192.168.1.30", + "syslog_port": 514, + "syslog_protocol": "tcp", + "loki_url": "http://192.168.1.221:3100/loki/api/v1/push", + "paperclip_metadata": True, + "paperclip_root": "~/.paperclip/instances/default", + "paperclip_context_path": "~/.paperclip/context.json", + "paperclip_api_base": None, + "paperclip_api_timeout": 1.5, +} + + +def main(argv: Optional[List[str]] = None) -> int: + parser = argparse.ArgumentParser( + prog="codex-usage-collector", + description="Collect changed AI coding session files and push them to a central Codex Usage Profiler host.", + ) + parser.add_argument("--config", help="Collector JSON config") + parser.add_argument("--once", action="store_true", help="Run one collection pass") + parser.add_argument("--dry-run", action="store_true", help="Scan and log without staging or pushing") + parser.add_argument("--print-default-config", action="store_true", help="Print a starter config") + args = parser.parse_args(argv) + + if args.print_default_config: + print(json.dumps(default_config(), indent=2, sort_keys=True)) + return 0 + + config = load_collector_config(args.config) + result = run_collection(config, dry_run=args.dry_run) + print(json.dumps(result, sort_keys=True)) + return 0 if not result.get("errors") else 2 + + +def default_config() -> Dict[str, Any]: + data = dict(DEFAULT_CONFIG) + data["tools"] = {tool: list(globs) for tool, globs in DEFAULT_TOOL_GLOBS.items()} + return data + + +def load_collector_config(path: Optional[str]) -> Dict[str, Any]: + config = dict(DEFAULT_CONFIG) + if path: + loaded = _read_json(Path(path).expanduser()) + if isinstance(loaded, dict): + config.update(loaded) + tools: Dict[str, List[str]] = {} + if config.get("include_defaults", True): + tools.update({tool: list(globs) for tool, globs in DEFAULT_TOOL_GLOBS.items()}) + for tool, globs_value in dict(config.get("tools") or {}).items(): + if isinstance(globs_value, str): + tools.setdefault(str(tool), []).append(globs_value) + elif isinstance(globs_value, list): + tools.setdefault(str(tool), []).extend(str(item) for item in globs_value) + config["tools"] = tools + config["collector_name"] = config.get("collector_name") or socket.gethostname().split(".")[0] + return config + + +def run_collection(config: Dict[str, Any], dry_run: bool = False) -> Dict[str, Any]: + started = time.time() + collector = str(config.get("collector_name") or socket.gethostname().split(".")[0]) + state_path = Path(str(config.get("state_path"))).expanduser() + staging_root = Path(str(config.get("staging_dir"))).expanduser() + log_path = Path(str(config.get("log_path"))).expanduser() + state = _read_json(state_path) + if not isinstance(state, dict): + state = {"files": {}} + state_files: Dict[str, Any] = dict(state.get("files") or {}) + candidate_rows = discover_candidates(config) + staged = 0 + skipped = 0 + oversized = 0 + vanished = 0 + errors: List[str] = [] + matched_by_tool: Dict[str, int] = {} + staged_by_tool: Dict[str, int] = {} + + for tool, path in candidate_rows: + matched_by_tool[tool] = matched_by_tool.get(tool, 0) + 1 + try: + stat = path.stat() + except FileNotFoundError: + vanished += 1 + continue + except OSError as exc: + errors.append(f"stat_failed:{path}:{type(exc).__name__}") + continue + max_bytes = int(config.get("max_file_bytes") or 0) + if max_bytes and stat.st_size > max_bytes: + oversized += 1 + continue + state_key = str(path) + fingerprint = {"size": stat.st_size, "mtime_ns": stat.st_mtime_ns} + if state_files.get(state_key) == fingerprint: + skipped += 1 + continue + if not dry_run: + try: + stage_file(staging_root, collector, tool, path, stat) + except FileNotFoundError: + vanished += 1 + continue + except OSError as exc: + errors.append(f"stage_failed:{path}:{type(exc).__name__}") + continue + state_files[state_key] = fingerprint + staged += 1 + staged_by_tool[tool] = staged_by_tool.get(tool, 0) + 1 + + if bool(config.get("paperclip_metadata", False)): + try: + metadata_result = stage_paperclip_metadata(staging_root, collector, config, dry_run=dry_run) + except OSError as exc: + errors.append(f"paperclip_metadata_failed:{type(exc).__name__}") + else: + if metadata_result: + metadata_key, metadata_fingerprint = metadata_result + if state_files.get(metadata_key) == metadata_fingerprint: + skipped += 1 + else: + if not dry_run: + state_files[metadata_key] = metadata_fingerprint + staged += 1 + staged_by_tool["paperclip_metadata"] = staged_by_tool.get("paperclip_metadata", 0) + 1 + + pushed = 0 + if staged and not dry_run: + push_error = push_staged(staging_root / collector, str(config.get("destination") or "")) + if push_error: + errors.append(push_error) + else: + pushed = staged + state["files"] = state_files + _write_json(state_path, state) + + event = { + "event": "collector_run", + "collector": collector, + "host": socket.gethostname(), + "matched_files": len(candidate_rows), + "matched_by_tool": matched_by_tool, + "staged_files": staged, + "staged_by_tool": staged_by_tool, + "skipped_files": skipped, + "oversized_files": oversized, + "vanished_files": vanished, + "pushed_files": pushed, + "dry_run": dry_run, + "duration_seconds": round(time.time() - started, 3), + "errors": errors, + } + write_event(log_path, event) + emit_syslog_json( + event, + host=str(config.get("syslog_host") or "") or None, + port=int(config.get("syslog_port") or 514), + tag="codex-usage-collector", + protocol=str(config.get("syslog_protocol") or "tcp"), + ) + emit_loki_json( + event, + url=str(config.get("loki_url") or "") or None, + labels={"service": "collector", "host": collector}, + ) + return event + + +def discover_candidates(config: Dict[str, Any]) -> List[Tuple[str, Path]]: + rows: List[Tuple[str, Path]] = [] + seen: set[Path] = set() + excludes = [str(item) for item in config.get("exclude") or []] + for tool, patterns in dict(config.get("tools") or {}).items(): + for pattern in patterns: + expanded = os.path.expanduser(os.path.expandvars(str(pattern))) + for match in glob.glob(expanded, recursive=True): + path = Path(match) + if not path.is_file(): + continue + if path.suffix.lower() not in {".jsonl", ".json"}: + continue + if should_exclude(path, excludes): + continue + resolved = path.resolve() + if resolved in seen: + continue + seen.add(resolved) + rows.append((str(tool), resolved)) + return sorted(rows, key=lambda item: (item[0], str(item[1]))) + + +def should_exclude(path: Path, patterns: Iterable[str]) -> bool: + text = str(path) + if "CodexBar" in text or "com.steipete.codexbar" in text: + return False + return any(fnmatch.fnmatch(text, os.path.expanduser(pattern)) for pattern in patterns) + + +def stage_file(staging_root: Path, collector: str, tool: str, path: Path, stat: os.stat_result) -> Path: + digest = hashlib.sha256(str(path).encode("utf-8")).hexdigest()[:16] + dest_dir = staging_root / collector / tool / digest + dest_dir.mkdir(parents=True, exist_ok=True) + dest = dest_dir / path.name + shutil.copy2(path, dest) + meta = { + "collector": collector, + "host": socket.gethostname(), + "tool": tool, + "source_path": str(path), + "source_path_sha256": hashlib.sha256(str(path).encode("utf-8")).hexdigest(), + "size": stat.st_size, + "mtime": stat.st_mtime, + "staged_at": int(time.time()), + } + paperclip = _paperclip_context_from_path(str(path)) + if paperclip: + meta["paperclip"] = paperclip + _write_json(dest.with_suffix(dest.suffix + ".meta.json"), meta) + return dest + + +def stage_paperclip_metadata( + staging_root: Path, + collector: str, + config: Dict[str, Any], + dry_run: bool = False, +) -> Optional[Tuple[str, Dict[str, str]]]: + snapshot = snapshot_paperclip_metadata(config) + if not snapshot: + return None + encoded = json.dumps(snapshot, sort_keys=True, separators=(",", ":")).encode("utf-8") + digest = hashlib.sha256(encoded).hexdigest() + state_key = f"paperclip_metadata:{snapshot.get('root') or collector}" + fingerprint = {"sha256": digest} + if not dry_run: + dest_dir = staging_root / collector / "paperclip_metadata" / digest[:16] + dest_dir.mkdir(parents=True, exist_ok=True) + _write_json(dest_dir / "paperclip-metadata.json", snapshot) + return state_key, fingerprint + + +def snapshot_paperclip_metadata(config: Dict[str, Any]) -> Optional[Dict[str, Any]]: + root = Path(str(config.get("paperclip_root") or "~/.paperclip/instances/default")).expanduser() + context_path = Path(str(config.get("paperclip_context_path") or "~/.paperclip/context.json")).expanduser() + context = _read_json(context_path) + if not root.exists() and not isinstance(context, dict): + return None + api_base = str(config.get("paperclip_api_base") or _api_base_from_context(context) or "http://127.0.0.1:3100").rstrip("/") + timeout = float(config.get("paperclip_api_timeout") or 1.5) + warnings: List[str] = [] + companies = _fetch_json_list(f"{api_base}/api/companies", timeout, warnings, "companies") + agents: List[Dict[str, Any]] = [] + projects: List[Dict[str, Any]] = [] + for company in companies: + company_id = str(company.get("id") or "") + if not company_id: + continue + agents.extend(_fetch_json_list(f"{api_base}/api/companies/{company_id}/agents", timeout, warnings, f"agents:{company_id}")) + projects.extend(_fetch_json_list(f"{api_base}/api/companies/{company_id}/projects", timeout, warnings, f"projects:{company_id}")) + snapshot = { + "schema": "codex-usage-profiler.paperclip-metadata.v1", + "root": str(root), + "api_base": api_base, + "captured_at": int(time.time()), + "context": _sanitize_paperclip_context(context), + "companies": [_pick_fields(row, ["id", "name", "status", "issuePrefix"]) for row in companies], + "agents": [_sanitize_agent(row) for row in agents], + "projects": [_sanitize_project(row) for row in projects], + "workspaces": _paperclip_workspace_summaries(root, companies), + "warnings": warnings, + } + if not snapshot["companies"] and not snapshot["agents"] and not snapshot["projects"] and not snapshot["workspaces"]: + return None + return snapshot + + +def push_staged(source: Path, destination: str) -> Optional[str]: + if not destination: + return "push_failed:missing_destination" + if not source.exists(): + return None + if _is_remote_destination(destination): + dest = destination.rstrip("/") + f"/{source.name}/" + proc = subprocess.run(["rsync", "-a", str(source) + "/", dest], text=True, capture_output=True, check=False) + if proc.returncode != 0: + return f"push_failed:rsync:{proc.stderr.strip() or proc.stdout.strip()}" + return None + dest_path = Path(destination).expanduser() / source.name + shutil.copytree(source, dest_path, dirs_exist_ok=True) + return None + + +def _api_base_from_context(context: Any) -> Optional[str]: + if not isinstance(context, dict): + return None + profiles = context.get("profiles") + current = context.get("currentProfile") + if isinstance(profiles, dict) and current in profiles and isinstance(profiles[current], dict): + value = profiles[current].get("apiBase") + if isinstance(value, str) and value: + return value + return None + + +def _fetch_json_list(url: str, timeout: float, warnings: List[str], label: str) -> List[Dict[str, Any]]: + try: + with urllib.request.urlopen(url, timeout=timeout) as response: + payload = json.loads(response.read().decode("utf-8")) + except (OSError, urllib.error.URLError, json.JSONDecodeError) as exc: + warnings.append(f"{label}:{type(exc).__name__}") + return [] + if not isinstance(payload, list): + return [] + return [item for item in payload if isinstance(item, dict)] + + +def _sanitize_paperclip_context(context: Any) -> Dict[str, Any]: + if not isinstance(context, dict): + return {} + return { + "currentProfile": context.get("currentProfile"), + "companyId": context.get("companyId"), + "agentId": context.get("agentId"), + "persona": context.get("persona"), + } + + +def _sanitize_agent(row: Dict[str, Any]) -> Dict[str, Any]: + adapter = row.get("adapterConfig") if isinstance(row.get("adapterConfig"), dict) else {} + env = adapter.get("env") if isinstance(adapter.get("env"), dict) else {} + codex_home = env.get("CODEX_HOME") if isinstance(env.get("CODEX_HOME"), dict) else {} + return { + "id": row.get("id"), + "companyId": row.get("companyId"), + "name": row.get("name"), + "title": row.get("title"), + "role": row.get("role"), + "status": row.get("status"), + "reportsTo": row.get("reportsTo"), + "adapterType": row.get("adapterType"), + "cwd": adapter.get("cwd"), + "codexHome": codex_home.get("value"), + "instructionsFilePath": adapter.get("instructionsFilePath"), + } + + +def _sanitize_project(row: Dict[str, Any]) -> Dict[str, Any]: + codebase = row.get("codebase") if isinstance(row.get("codebase"), dict) else {} + return { + "id": row.get("id"), + "companyId": row.get("companyId"), + "name": row.get("name"), + "status": row.get("status"), + "leadAgentId": row.get("leadAgentId"), + "urlKey": row.get("urlKey"), + "workspaceId": codebase.get("workspaceId"), + "localFolder": codebase.get("localFolder"), + "managedFolder": codebase.get("managedFolder"), + "effectiveLocalFolder": codebase.get("effectiveLocalFolder"), + } + + +def _pick_fields(row: Dict[str, Any], fields: List[str]) -> Dict[str, Any]: + return {field: row.get(field) for field in fields if field in row} + + +def _paperclip_workspace_summaries(root: Path, companies: List[Dict[str, Any]]) -> List[Dict[str, Any]]: + workspaces = root / "workspaces" + if not workspaces.exists(): + return [] + prefix_company = { + str(company.get("issuePrefix") or "").upper(): str(company.get("id") or "") + for company in companies + if company.get("issuePrefix") and company.get("id") + } + rows: List[Dict[str, Any]] = [] + for workspace in sorted(workspaces.iterdir()): + if not workspace.is_dir(): + continue + names = [item.name for item in workspace.iterdir() if item.is_file()] + if not names: + continue + staff = _staff_from_workspace_names(names) + issue_prefixes = sorted(_issue_prefixes_from_names(names, set(prefix_company))) + company_id = prefix_company.get(issue_prefixes[0]) if issue_prefixes else None + task_ids = sorted(_task_ids_from_names(names))[:20] + rows.append( + { + "id": workspace.name, + "path": str(workspace), + "staffLabel": staff, + "companyId": company_id, + "issuePrefixes": issue_prefixes, + "taskIds": task_ids, + "fileCount": len(names), + } + ) + return rows + + +def _staff_from_workspace_names(names: List[str]) -> Optional[str]: + aliases = { + "ceo": "CEO", + "cto": "CTO", + "cmo": "CMO", + "cfo": "CFO", + "coo": "COO", + "sre": "SRE", + "qa": "QA", + "devops": "DevOps", + "security": "Security", + "growth": "Growth", + } + counts: Dict[str, int] = {} + for name in names: + match = re.match(r"([a-z][a-z0-9]+)[-_]", name.lower()) + if match: + counts[match.group(1)] = counts.get(match.group(1), 0) + 1 + if not counts: + return None + key = sorted(counts.items(), key=lambda item: (-item[1], item[0]))[0][0] + return aliases.get(key, key.replace("_", " ").replace("-", " ").title()) + + +def _issue_prefixes_from_names(names: List[str], known_prefixes: set[str]) -> set[str]: + prefixes = set() + for name in names: + for match in re.finditer(r"\b([a-z]{2,10})[-_]?(\d{1,6})\b", name, re.IGNORECASE): + prefix = match.group(1).upper() + if not known_prefixes or prefix in known_prefixes: + prefixes.add(prefix) + return prefixes + + +def _task_ids_from_names(names: List[str]) -> set[str]: + ids = set() + for name in names: + for match in re.finditer(r"\b([A-Z]{2,10})[-_]?(\d{1,6})\b", name, re.IGNORECASE): + ids.add(f"{match.group(1).upper()}-{match.group(2)}") + return ids + + +def _paperclip_context_from_path(path: str) -> Dict[str, str]: + result: Dict[str, str] = {} + match = re.search(r"/(?:\.paperclip|paperclip)/instances/([^/]+)/companies/([^/]+)(?:/agents/([^/]+))?/codex-home/", path) + if match: + result["instance_id"] = match.group(1) + result["company_id"] = match.group(2) + if match.group(3): + result["agent_id"] = match.group(3) + match = re.search(r"/(?:\.paperclip|paperclip)/instances/([^/]+)/projects/([^/]+)/([^/]+)/", path) + if match: + result["instance_id"] = match.group(1) + result["company_id"] = match.group(2) + result["project_id"] = match.group(3) + match = re.search(r"/(?:\.paperclip|paperclip)/instances/([^/]+)/workspaces/([^/]+)(?:/|$)", path) + if match: + result["instance_id"] = match.group(1) + result["workspace_id"] = match.group(2) + return result + + +def write_event(log_path: Path, event: Dict[str, Any]) -> None: + log_path.parent.mkdir(parents=True, exist_ok=True) + with log_path.open("a", encoding="utf-8") as fh: + fh.write(json.dumps(event, sort_keys=True) + "\n") + + +def _is_remote_destination(destination: str) -> bool: + head = destination.split("/", 1)[0] + return ":" in head + + +def _read_json(path: Path) -> Any: + try: + return json.loads(path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + return {} + + +def _write_json(path: Path, data: Any) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_suffix(path.suffix + ".tmp") + tmp.write_text(json.dumps(data, indent=2, sort_keys=True), encoding="utf-8") + tmp.replace(path) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/codex_usage_profiler/config.py b/src/codex_usage_profiler/config.py index 7ecb6e4..c944dd0 100644 --- a/src/codex_usage_profiler/config.py +++ b/src/codex_usage_profiler/config.py @@ -12,9 +12,14 @@ class Config: path_aliases: Dict[str, str] = field(default_factory=dict) client_aliases: Dict[str, str] = field(default_factory=dict) task_aliases: Dict[str, str] = field(default_factory=dict) + paperclip_company_aliases: Dict[str, str] = field(default_factory=dict) + paperclip_project_aliases: Dict[str, str] = field(default_factory=dict) + paperclip_agent_aliases: Dict[str, str] = field(default_factory=dict) rates: Dict[str, Dict[str, float]] = field(default_factory=dict) + plan_prices_usd: Dict[str, float] = field(default_factory=dict) dollars_per_credit: Optional[float] = None monthly_plan_price_usd: Optional[float] = None + projection_days: int = 30 codexbar_enabled: bool = True paperclip_enabled: bool = True paperclip_root: str = "~/.paperclip/instances/default" @@ -33,9 +38,18 @@ def load_config(path: Optional[str]) -> Config: path_aliases=dict(data.get("path_aliases") or {}), client_aliases=dict(data.get("client_aliases") or {}), task_aliases=dict(data.get("task_aliases") or {}), + paperclip_company_aliases=dict(data.get("paperclip_company_aliases") or {}), + paperclip_project_aliases=dict(data.get("paperclip_project_aliases") or {}), + paperclip_agent_aliases=dict(data.get("paperclip_agent_aliases") or {}), rates=dict(data.get("rates") or {}), + plan_prices_usd={ + str(name): float(value) + for name, value in dict(data.get("plan_prices_usd") or {}).items() + if isinstance(value, (int, float)) + }, dollars_per_credit=_num(data.get("dollars_per_credit")), monthly_plan_price_usd=_num(data.get("monthly_plan_price_usd")), + projection_days=int(data.get("projection_days") or 30), codexbar_enabled=bool(data.get("codexbar_enabled", True)), paperclip_enabled=bool(data.get("paperclip_enabled", True)), paperclip_root=str(data.get("paperclip_root") or "~/.paperclip/instances/default"), diff --git a/src/codex_usage_profiler/dashboard.py b/src/codex_usage_profiler/dashboard.py index d07e8b4..5a67bfe 100644 --- a/src/codex_usage_profiler/dashboard.py +++ b/src/codex_usage_profiler/dashboard.py @@ -3,6 +3,7 @@ import argparse import json import mimetypes +import socket import sys from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from importlib import resources @@ -10,6 +11,8 @@ from typing import Optional from urllib.parse import unquote, urlparse +from .syslog_util import emit_loki_json, emit_syslog_json + STATIC_PACKAGE = "codex_usage_profiler" STATIC_DIR = "dashboard_static" @@ -21,6 +24,9 @@ def _static_root(): class DashboardHandler(BaseHTTPRequestHandler): report_path: Path + syslog_host: Optional[str] = None + syslog_port: int = 514 + loki_url: Optional[str] = None def do_GET(self) -> None: parsed = urlparse(self.path) @@ -36,7 +42,31 @@ def do_GET(self) -> None: self._serve_static(path.lstrip("/")) def log_message(self, format: str, *args) -> None: # noqa: A002 - sys.stderr.write("dashboard: " + (format % args) + "\n") + message = format % args + sys.stderr.write("dashboard: " + message + "\n") + emit_syslog_json( + { + "event": "dashboard_access", + "host": socket.gethostname(), + "client": self.client_address[0] if self.client_address else "unknown", + "request": getattr(self, "requestline", ""), + "message": message, + }, + host=self.syslog_host, + port=self.syslog_port, + tag="codex-usage-dashboard", + ) + emit_loki_json( + { + "event": "dashboard_access", + "host": socket.gethostname(), + "client": self.client_address[0] if self.client_address else "unknown", + "request": getattr(self, "requestline", ""), + "message": message, + }, + url=self.loki_url, + labels={"service": "dashboard", "host": socket.gethostname()}, + ) def _serve_report(self) -> None: try: @@ -85,13 +115,23 @@ def _send_error(self, status: int, message: str) -> None: self.wfile.write(data) -def make_server(report_path: str | Path, host: str = "127.0.0.1", port: int = 8765) -> ThreadingHTTPServer: +def make_server( + report_path: str | Path, + host: str = "127.0.0.1", + port: int = 8765, + syslog_host: Optional[str] = None, + syslog_port: int = 514, + loki_url: Optional[str] = None, +) -> ThreadingHTTPServer: report = Path(report_path).expanduser().resolve() class BoundDashboardHandler(DashboardHandler): pass BoundDashboardHandler.report_path = report + BoundDashboardHandler.syslog_host = syslog_host + BoundDashboardHandler.syslog_port = syslog_port + BoundDashboardHandler.loki_url = loki_url return ThreadingHTTPServer((host, port), BoundDashboardHandler) @@ -103,9 +143,12 @@ def main(argv: Optional[list[str]] = None) -> int: parser.add_argument("--report", default="samples/demo-report.json", help="Profiler JSON report to serve") parser.add_argument("--host", default="127.0.0.1", help="Bind host. Use 0.0.0.0 for Tailscale/LAN access") parser.add_argument("--port", type=int, default=8765, help="Bind port") + parser.add_argument("--syslog-host", help="Optional syslog host for dashboard access events") + parser.add_argument("--syslog-port", type=int, default=514) + parser.add_argument("--loki-url", help="Optional Loki push API URL for dashboard access events") args = parser.parse_args(argv) - server = make_server(args.report, args.host, args.port) + server = make_server(args.report, args.host, args.port, args.syslog_host, args.syslog_port, args.loki_url) bound_host, bound_port = server.server_address print(f"Codex Usage Profiler dashboard: http://{bound_host}:{bound_port}/") print(f"Report: {Path(args.report).expanduser().resolve()}") diff --git a/src/codex_usage_profiler/dashboard_static/app.js b/src/codex_usage_profiler/dashboard_static/app.js index af1e38e..518a7d2 100644 --- a/src/codex_usage_profiler/dashboard_static/app.js +++ b/src/codex_usage_profiler/dashboard_static/app.js @@ -46,6 +46,7 @@ "waste-drivers": "Ranked recurring review candidates. Click a row to filter the session table to that pattern. Candidate rows can overlap.", coverage: "Attribution coverage shows how much filtered usage can be tied to client, project, staff, and task. Click unknown buckets to investigate gaps.", confidence: "Confidence is the weakest key attribution signal across client, project, staff, and task. Lower confidence means the profiler needs better attribution evidence.", + "company-spend": "Paperclip company spend groups filtered sessions by Paperclip company and day. Projected cost uses the filtered observed span, so treat it as directional plan-selection evidence.", projection: "Cleanup projection estimates directional savings from de-duplicated sessions in the top review-candidate drivers.", notifications: "Notification hooks are not enabled in this local-only dashboard yet.", settings: "Settings will hold report paths, privacy defaults, and quota assumptions in a later version." @@ -504,6 +505,71 @@ return Object.keys(map).map(function (key) { return map[key]; }).sort(function (a, b) { return b.tokens - a.tokens; }); } + function dayString(session) { + var date = startDate(session); + return date ? date.toISOString().slice(0, 10) : "unknown"; + } + + function dateSpanDays(days) { + days = (days || []).filter(function (day) { return day !== "unknown"; }).sort(); + if (!days.length) return 1; + var first = new Date(days[0] + "T00:00:00Z").getTime(); + var last = new Date(days[days.length - 1] + "T00:00:00Z").getTime(); + if (!isFinite(first) || !isFinite(last)) return 1; + return Math.max(1, Math.round((last - first) / 86400000) + 1); + } + + function companySpendModel(sessions, report, metric) { + var valueFor = metric === "tokens" ? tokens : cost; + var projectionDays = Number((report.plan_analysis && report.plan_analysis.projection_days) || (report.paperclip_spend && report.paperclip_spend.projection_days) || 30); + var totals = {}; + var days = {}; + var dayCompany = {}; + (sessions || []).forEach(function (session) { + var company = attrLabel(session.paperclip_company); + if (company === "unknown") return; + var day = dayString(session); + var value = valueFor(session); + if (!totals[company]) totals[company] = { company: company, sessions: 0, tokens: 0, cost: 0, days: {} }; + totals[company].sessions += 1; + totals[company].tokens += tokens(session); + totals[company].cost += cost(session); + totals[company].days[day] = true; + days[day] = true; + var key = day + "|" + company; + if (!dayCompany[key]) dayCompany[key] = { day: day, company: company, sessions: 0, tokens: 0, cost: 0, value: 0 }; + dayCompany[key].sessions += 1; + dayCompany[key].tokens += tokens(session); + dayCompany[key].cost += cost(session); + dayCompany[key].value += value; + }); + var totalRows = Object.keys(totals).map(function (company) { + var row = totals[company]; + var activeDays = Object.keys(row.days).length; + var span = dateSpanDays(Object.keys(row.days)); + row.activeDays = activeDays; + row.observedSpanDays = span; + row.projectedCost = span ? row.cost / span * projectionDays : 0; + row.projectedTokens = span ? row.tokens / span * projectionDays : 0; + return row; + }).sort(function (a, b) { return b[metric] - a[metric]; }); + var topCompanies = {}; + totalRows.slice(0, 5).forEach(function (row) { topCompanies[row.company] = true; }); + var dayRows = Object.keys(days).sort().slice(-14).map(function (day) { + var companies = {}; + var total = 0; + Object.keys(dayCompany).forEach(function (key) { + var row = dayCompany[key]; + if (row.day !== day) return; + var label = topCompanies[row.company] ? row.company : "Other"; + companies[label] = (companies[label] || 0) + row.value; + total += row.value; + }); + return { day: day, companies: companies, total: total }; + }); + return { metric: metric, projectionDays: projectionDays, totals: totalRows, days: dayRows }; + } + function outcomeBucket(session, findingIndex) { if (isWaste(session, findingIndex || {})) return "Waste"; if (isUseful(session)) return "Useful"; @@ -588,11 +654,11 @@ } function layoutFlowModel(model, width, height) { - width = Math.max(360, Number(width) || 520); + width = Math.max(300, Number(width) || 520); var maxNodes = model.columns.reduce(function (max, column) { return Math.max(max, column.nodes.length); }, 0); var requiredHeight = 46 + maxNodes * 36 + Math.max(0, maxNodes - 1) * 8; height = Math.max(220, Number(height) || 300, requiredHeight); - var nodeWidth = Math.min(144, Math.max(88, width * 0.16)); + var nodeWidth = Math.min(144, Math.max(68, width * 0.17)); var stageGap = model.columns.length > 1 ? (width - nodeWidth) / (model.columns.length - 1) : 0; var topPad = 34; var bottomPad = 12; @@ -1035,6 +1101,71 @@ commitFilterChange(); } + function isCardFilterActive(kind) { + var filters = app.state.filters; + if (kind === "flow") { + return Boolean( + filters.clients.length || + filters.projects.length || + filters.staff.length || + filters.outcomes.length || + filters.outcomeBucket || + filters.sessionIds.length + ); + } + if (kind === "timeline") return Boolean(filters.brushStartTime || filters.brushEndTime || app.state.hiddenOutcomes.length); + if (kind === "company") return Boolean(filters.companies.length || filters.brushStartTime || filters.brushEndTime); + if (kind === "heatmap") return Boolean(filters.weekdays.length || filters.hourStart !== "" || filters.hourEnd !== ""); + if (kind === "waste") return Boolean(filters.waste !== "all" || filters.wasteKind); + if (kind === "coverage") return Boolean(filters.attributionCoverage); + if (kind === "projection") return Boolean(filters.waste === "any" && !filters.wasteKind); + return false; + } + + function resetCardFilter(kind) { + var filters = app.state.filters; + if (kind === "flow") { + var hadOutcomeBucket = Boolean(filters.outcomeBucket); + filters.clients = []; + filters.projects = []; + filters.staff = []; + filters.outcomes = []; + filters.outcomeBucket = ""; + filters.sessionIds = []; + if (hadOutcomeBucket && filters.waste === "any" && !filters.wasteKind) filters.waste = "all"; + } else if (kind === "timeline") { + filters.brushStartTime = ""; + filters.brushEndTime = ""; + app.state.hiddenOutcomes = []; + } else if (kind === "company") { + filters.companies = []; + filters.brushStartTime = ""; + filters.brushEndTime = ""; + } else if (kind === "heatmap") { + filters.weekdays = []; + filters.hourStart = ""; + filters.hourEnd = ""; + } else if (kind === "waste") { + filters.waste = "all"; + filters.wasteKind = ""; + } else if (kind === "coverage") { + filters.attributionCoverage = ""; + } else if (kind === "projection") { + if (filters.waste === "any" && !filters.wasteKind) filters.waste = "all"; + } + commitFilterChange(); + } + + function updateCardResetButtons() { + Array.prototype.slice.call(document.querySelectorAll(".card-reset[data-reset-card]")).forEach(function (button) { + var kind = button.getAttribute("data-reset-card"); + var active = isCardFilterActive(kind); + button.disabled = !active; + button.classList.toggle("active", active); + button.setAttribute("aria-disabled", active ? "false" : "true"); + }); + } + function renderFlow(sessions, state) { var container = document.getElementById("spend-flow"); clear(container); @@ -1369,6 +1500,82 @@ }); } + function renderCompanySpend(sessions, report, state) { + var container = document.getElementById("company-spend"); + clear(container); + var metric = state.metric === "tokens" ? "tokens" : "cost"; + var model = companySpendModel(sessions, report || {}, metric); + if (!model.totals.length) { + container.appendChild(el("p", { class: "muted" }, ["No Paperclip company sessions in this filter."])); + return; + } + var maxDay = Math.max.apply(Math, model.days.map(function (row) { return row.total; }).concat([1])); + var maxTotal = Math.max.apply(Math, model.totals.slice(0, 5).map(function (row) { return row[metric]; }).concat([1])); + var planPrice = Number((report.plan_analysis && report.plan_analysis.monthly_plan_price_usd) || 0); + var topWrap = el("div", { class: "company-spend-top" }); + model.totals.slice(0, 5).forEach(function (row) { + var projected = metric === "tokens" ? row.projectedTokens : row.projectedCost; + var value = metric === "tokens" ? row.tokens : row.cost; + var button = el("button", { + type: "button", + class: sameArrayValues(app.state.filters.companies, [row.company]) ? "active" : "", + style: "--bar:" + Math.max(4, value / maxTotal * 100).toFixed(1) + "%", + "aria-pressed": sameArrayValues(app.state.filters.companies, [row.company]) ? "true" : "false", + "aria-label": "Filter Paperclip company " + row.company + }, [ + el("strong", { title: row.company }, [row.company]), + el("span", {}, [formatMetricValue(value, metric) + " observed"]), + el("span", {}, [formatMetricValue(projected, metric) + " projected " + model.projectionDays + "d"]) + ]); + button.addEventListener("click", function () { + app.state.filters.companies = sameArrayValues(app.state.filters.companies, [row.company]) ? [] : [row.company]; + commitFilterChange(); + }); + topWrap.appendChild(button); + }); + container.appendChild(topWrap); + var dayWrap = el("div", { class: "company-day-bars" }); + model.days.forEach(function (row) { + var activeDay = app.state.filters.brushStartTime === row.day + "T00:00:00.000Z" && app.state.filters.brushEndTime === row.day + "T23:59:59.999Z"; + var day = el("button", { + type: "button", + class: "company-day" + (activeDay ? " active" : ""), + title: row.day + " " + formatMetricValue(row.total, metric), + "aria-pressed": activeDay ? "true" : "false", + "aria-label": (activeDay ? "Clear day " : "Filter day ") + row.day + }, [ + el("span", { class: "day-label" }, [row.day.slice(5)]) + ]); + var stack = el("span", { class: "day-stack", style: "--height:" + Math.max(3, row.total / maxDay * 100).toFixed(1) + "%" }); + Object.keys(row.companies).sort(function (a, b) { return row.companies[b] - row.companies[a]; }).forEach(function (company, idx) { + stack.appendChild(el("span", { + class: "company-segment seg-" + (idx % 6), + style: "height:" + (row.companies[company] / Math.max(1, row.total) * 100).toFixed(1) + "%", + title: company + " " + formatMetricValue(row.companies[company], metric) + })); + }); + day.addEventListener("click", function () { + if (activeDay) { + app.state.filters.brushStartTime = ""; + app.state.filters.brushEndTime = ""; + } else { + app.state.filters.brushStartTime = row.day + "T00:00:00.000Z"; + app.state.filters.brushEndTime = row.day + "T23:59:59.999Z"; + } + commitFilterChange(); + }); + day.appendChild(stack); + dayWrap.appendChild(day); + }); + container.appendChild(dayWrap); + if (planPrice) { + var projectedCost = model.totals.reduce(function (sum, row) { return sum + row.projectedCost; }, 0); + container.appendChild(el("p", { class: "company-plan-note" }, [ + "Filtered projected rate-card cost " + formatCost(projectedCost) + " vs plan price " + formatCost(planPrice) + " (" + formatPercent(planPrice ? projectedCost / planPrice * 100 : null) + ")." + ])); + } + } + function renderWasteDrivers(sessions, report) { var container = document.getElementById("waste-drivers"); clear(container); @@ -1829,11 +2036,13 @@ renderKpis(summary); renderFlow(flowSessions, app.state); renderTimeline(app.filtered, app.report); + renderCompanySpend(app.filtered, app.report, app.state); renderHeatmap(app.filtered); renderWasteDrivers(app.filtered, app.report); renderCoverage(app.filtered); renderProjection(app.filtered, app.report); renderChips(app.state); + updateCardResetButtons(); renderTable(app.filtered, app.state, app.report); renderDrawer(app.filtered, app.state, app.report); var query = encodeFilters(app.state); @@ -1857,6 +2066,12 @@ }); document.getElementById("density-select").addEventListener("change", renderApp); document.getElementById("reduction-select").addEventListener("change", renderApp); + Array.prototype.slice.call(document.querySelectorAll(".card-reset[data-reset-card]")).forEach(function (button) { + button.addEventListener("click", function () { + if (button.disabled) return; + resetCardFilter(button.getAttribute("data-reset-card")); + }); + }); bindBrushEvents(); document.getElementById("prev-page").addEventListener("click", function () { app.state.page = Math.max(1, app.state.page - 1); @@ -1966,6 +2181,13 @@ document.body.classList.add("drawer-hidden"); renderApp(); }); + var resizeTimer = null; + window.addEventListener("resize", function () { + clearTimeout(resizeTimer); + resizeTimer = setTimeout(function () { + renderApp(); + }, 60); + }); } function copyText(text) { @@ -2172,6 +2394,7 @@ wasteDrivers: wasteDrivers, coverageStats: coverageStats, hourlyBuckets: hourlyBuckets, + companySpendModel: companySpendModel, timeExtent: timeExtent, sortSessions: sortSessions, normalizeSession: normalizeSession, diff --git a/src/codex_usage_profiler/dashboard_static/index.html b/src/codex_usage_profiler/dashboard_static/index.html index 20b6870..40ac8a8 100644 --- a/src/codex_usage_profiler/dashboard_static/index.html +++ b/src/codex_usage_profiler/dashboard_static/index.html @@ -128,13 +128,16 @@

Active filters

Spend Flow

Client -> Project -> Staff -> Outcome

- +
+ + +
@@ -154,6 +157,7 @@

Usage Over Time (Hourly) @@ -164,9 +168,20 @@

Usage Over Time (Hourly)

+

Observed and projected by company

+ + + +
+

Usage Heatmap (Tokens)

+
LowerHigher
@@ -174,6 +189,7 @@

Usage Heatmap (Tokens)

+
PatternSessionsCandidate Cost
@@ -182,6 +198,7 @@

Top Review Candidates

+
View unknown buckets @@ -197,6 +214,7 @@

Cleanup Projection
How projection is calculated diff --git a/src/codex_usage_profiler/dashboard_static/styles.css b/src/codex_usage_profiler/dashboard_static/styles.css index 76caf4b..ed8a7b6 100644 --- a/src/codex_usage_profiler/dashboard_static/styles.css +++ b/src/codex_usage_profiler/dashboard_static/styles.css @@ -151,7 +151,14 @@ h3 { display: flex; align-items: center; gap: 10px; - min-width: 455px; + flex: 0 1 520px; + min-width: 0; +} + +.brand h1 { + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; } .brand-mark { @@ -176,6 +183,9 @@ h3 { background: rgba(98, 200, 97, 0.18); color: #bbf3b8; white-space: nowrap; + max-width: 210px; + overflow: hidden; + text-overflow: ellipsis; } .source-button { @@ -325,6 +335,38 @@ body.filters-hidden .show-filters { justify-content: space-between; gap: 12px; min-height: 34px; + min-width: 0; + flex-wrap: wrap; +} + +.panel-head > * { + min-width: 0; +} + +.panel-actions { + display: flex; + align-items: center; + justify-content: flex-end; + gap: 6px; + min-width: 0; + flex-wrap: wrap; +} + +.card-reset { + min-height: 30px; + padding: 0 8px; + color: #b8dfff; + background: rgba(70, 169, 255, 0.08); +} + +.card-reset:disabled { + opacity: 0.42; + cursor: default; +} + +.card-reset.active { + border-color: var(--focus); + background: rgba(70, 169, 255, 0.18); } .filter-form { @@ -604,6 +646,8 @@ body.filters-hidden .app-shell { display: flex; align-items: center; gap: 6px; + min-width: 0; + flex-wrap: wrap; } .chart-actions button { @@ -759,6 +803,12 @@ body.filters-hidden .app-shell { line-height: 1.15; } +.flow-sankey .flow-node strong, +.flow-sankey .flow-node span { + white-space: normal; + overflow-wrap: anywhere; +} + .flow-has-highlight .flow-link, .flow-has-highlight .flow-node { opacity: 0.22; @@ -931,10 +981,140 @@ body.filters-hidden .app-shell { background: #7f8b93; } +.company-spend { + display: grid; + gap: 12px; + margin-top: 8px; +} + +.company-spend-top { + display: grid; + grid-template-columns: repeat(5, minmax(0, 1fr)); + gap: 8px; +} + +.company-spend-top button { + position: relative; + display: grid; + gap: 4px; + min-height: 74px; + padding: 8px; + overflow: hidden; + text-align: left; + background: #0b161e; +} + +.company-spend-top button::before { + content: ""; + position: absolute; + inset: auto 0 0; + height: var(--bar, 0%); + background: linear-gradient(180deg, rgba(70, 169, 255, 0.08), rgba(56, 216, 232, 0.28)); +} + +.company-spend-top strong, +.company-spend-top span { + position: relative; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.company-spend-top span { + color: var(--muted); + font-size: 11px; +} + +.company-spend-top button.active { + border-color: var(--focus); +} + +.company-day-bars { + display: flex; + align-items: end; + gap: 4px; + min-height: 116px; + padding: 10px 10px 22px; + border: 1px solid var(--line-soft); + border-radius: 6px; + background: + linear-gradient(180deg, transparent 49%, rgba(255, 255, 255, 0.05) 49% 50%, transparent 50%), + #071119; +} + +.company-day { + position: relative; + display: grid; + align-items: end; + justify-items: center; + flex: 1 1 26px; + min-width: 24px; + height: 92px; + padding: 0; + border: 0; + background: transparent; +} + +.company-day.active { + outline: 2px solid var(--focus); + outline-offset: 2px; +} + +.day-stack { + display: flex; + flex-direction: column-reverse; + width: 100%; + height: var(--height, 3%); + min-height: 3px; + border-radius: 3px 3px 0 0; + overflow: hidden; + background: #1c2e3d; +} + +.day-label { + position: absolute; + transform: translateY(17px); + color: var(--muted); + font-size: 10px; +} + +.company-segment { + min-height: 1px; +} + +.seg-0 { + background: var(--blue); +} + +.seg-1 { + background: var(--green); +} + +.seg-2 { + background: var(--yellow); +} + +.seg-3 { + background: var(--violet); +} + +.seg-4 { + background: var(--cyan); +} + +.seg-5 { + background: var(--red); +} + +.company-plan-note { + color: var(--muted); + font-size: 12px; +} + .heatmap { display: grid; - grid-template-columns: 28px repeat(24, minmax(7px, 1fr)); - gap: 3px; + grid-template-columns: 24px repeat(24, minmax(5px, 1fr)); + gap: 2px; margin-top: 8px; align-items: stretch; } @@ -1029,6 +1209,7 @@ body.filters-hidden .app-shell { grid-template-columns: 128px 1fr; gap: 12px; align-items: center; + min-width: 0; } .donut { @@ -1071,6 +1252,14 @@ body.filters-hidden .app-shell { justify-content: space-between; gap: 10px; color: var(--muted); + min-width: 0; +} + +.coverage-list span, +.coverage-list strong, +.projection span { + min-width: 0; + overflow-wrap: anywhere; } .coverage-list button { @@ -1483,7 +1672,7 @@ body.filters-hidden.drawer-hidden .app-shell { color: var(--muted); } -@media (max-width: 1380px) { +@media (max-width: 1500px) { .kpi-strip { grid-template-columns: repeat(3, minmax(0, 1fr)); } @@ -1505,6 +1694,12 @@ body.filters-hidden.drawer-hidden .app-shell { } } +@media (max-width: 1380px) { + .flow-sankey { + height: 320px; + } +} + @media (max-width: 1120px) { .app-shell { grid-template-columns: 236px minmax(0, 1fr); @@ -1527,6 +1722,7 @@ body.filters-hidden.drawer-hidden .app-shell { .brand { min-width: 0; + flex-wrap: wrap; } .command-row { @@ -1567,6 +1763,10 @@ body.filters-hidden.drawer-hidden .app-shell { .flow-grid { grid-template-columns: 1fr 1fr; } + + .company-spend-top { + grid-template-columns: repeat(2, minmax(0, 1fr)); + } } @media (max-width: 560px) { @@ -1574,6 +1774,24 @@ body.filters-hidden.drawer-hidden .app-shell { font-size: 12px; } + h1 { + white-space: normal; + line-height: 1.15; + } + + .source-pill, + .live-status { + max-width: 100%; + white-space: normal; + } + + .panel-head, + .chart-actions, + .panel-actions { + align-items: flex-start; + justify-content: flex-start; + } + .kpi-strip { display: grid; } @@ -1588,6 +1806,19 @@ body.filters-hidden.drawer-hidden .app-shell { grid-template-columns: 1fr; } + .company-spend-top { + grid-template-columns: 1fr; + } + + .company-day-bars { + overflow-x: auto; + justify-content: flex-start; + } + + .company-day { + flex: 0 0 26px; + } + .timeline { height: 190px; overflow-x: auto; diff --git a/src/codex_usage_profiler/ingest.py b/src/codex_usage_profiler/ingest.py index 3a7175f..a96b9aa 100644 --- a/src/codex_usage_profiler/ingest.py +++ b/src/codex_usage_profiler/ingest.py @@ -48,6 +48,7 @@ def parse_logs(paths: Optional[List[str]] = None, since: Optional[str] = None) - def parse_log(path: Path) -> SessionRecord: record = SessionRecord(session_id=path.stem, path=str(path)) + _apply_collector_sidecar(record, path) current_usage = empty_usage() event_counts: Counter[str] = Counter() tool_counts: Counter[str] = Counter() @@ -96,11 +97,20 @@ def parse_log(path: Path) -> SessionRecord: current_usage = usage record.usage_known = True + _apply_generic_metadata(record, event) + usage = _extract_usage(event) + if usage: + usage_total = usage.get("total_tokens", 0) + if usage_total >= current_usage.get("total_tokens", 0): + current_usage = usage + record.usage_known = True + request_text = _find_user_text(event) if request_text and not record.first_request_hash: record.first_request_hash = stable_hash(request_text) record.first_request_snippet = bounded_snippet(request_text) - record.first_request_features = _extract_request_features(request_text) + for key, value in _extract_request_features(request_text).items(): + record.first_request_features.setdefault(key, value) tool_name, arguments = _find_tool_call(event) if tool_name: @@ -160,6 +170,63 @@ def _apply_turn_context(record: SessionRecord, payload: Dict[str, Any]) -> None: record.workspace_roots = [r for r in (scalar_to_string(item) for item in roots) if r] +def _apply_generic_metadata(record: SessionRecord, event: Dict[str, Any]) -> None: + record.cwd = scalar_to_string(event.get("cwd") or event.get("working_dir") or event.get("workingDirectory")) or record.cwd + record.model = scalar_to_string(event.get("model") or event.get("model_name")) or record.model + record.source = scalar_to_string(event.get("source") or event.get("client") or event.get("app")) or record.source + record.originator = scalar_to_string(event.get("originator") or event.get("tool")) or record.originator + roots = event.get("workspace_roots") or event.get("workspaceRoots") + if isinstance(roots, list): + record.workspace_roots = [r for r in (scalar_to_string(item) for item in roots) if r] or record.workspace_roots + _apply_paperclip_metadata(record, event) + payload = event.get("payload") + if isinstance(payload, dict): + _apply_paperclip_metadata(record, payload) + + +def _apply_collector_sidecar(record: SessionRecord, path: Path) -> None: + meta = _read_json(path.with_suffix(path.suffix + ".meta.json")) + if not isinstance(meta, dict): + return + record.source_path = scalar_to_string(meta.get("source_path")) or record.source_path + record.collector = scalar_to_string(meta.get("collector")) or record.collector + record.collector_tool = scalar_to_string(meta.get("tool")) or record.collector_tool + if record.collector_tool and not record.source: + record.source = record.collector_tool + if record.source_path and record.source_path not in record.workspace_roots: + record.workspace_roots.append(record.source_path) + _apply_paperclip_metadata(record, meta) + + +def _apply_paperclip_metadata(record: SessionRecord, event: Dict[str, Any]) -> None: + candidates: List[Dict[str, Any]] = [event] + paperclip = event.get("paperclip") or event.get("paperclip_metadata") or event.get("paperclipMetadata") + if isinstance(paperclip, dict): + candidates.append(paperclip) + mappings = { + "company": ["paperclip_company", "paperclipCompany", "company", "company_name", "companyName"], + "company_id": ["paperclip_company_id", "paperclipCompanyId", "company_id", "companyId"], + "project": ["paperclip_project", "paperclipProject", "project", "project_name", "projectName"], + "project_id": ["paperclip_project_id", "paperclipProjectId", "project_id", "projectId"], + "agent": ["paperclip_agent", "paperclipAgent", "agent", "agent_name", "agentName"], + "agent_id": ["paperclip_agent_id", "paperclipAgentId", "agent_id", "agentId", "staff_id", "staffId"], + "workspace_id": ["paperclip_workspace_id", "paperclipWorkspaceId", "workspace_id", "workspaceId"], + "staff": ["paperclip_staff", "paperclipStaff", "staff", "staff_name", "staffName"], + "task": ["paperclip_task", "paperclipTask", "task", "task_title", "taskTitle"], + "issue": ["paperclip_issue", "paperclipIssue", "issue", "issue_key", "issueKey"], + "queue": ["paperclip_queue", "paperclipQueue", "queue"], + } + for source in candidates: + for feature_key, field_names in mappings.items(): + if record.first_request_features.get(feature_key): + continue + for field_name in field_names: + value = scalar_to_string(source.get(field_name)) + if value: + record.first_request_features[feature_key] = bounded_snippet(value, 120) + break + + def _extract_usage(payload: Dict[str, Any]) -> Optional[Dict[str, int]]: info = payload.get("info") usage = None @@ -174,8 +241,18 @@ def _extract_usage(payload: Dict[str, Any]) -> Optional[Dict[str, int]]: value = usage.get(key) if isinstance(value, (int, float)): result[key] = int(value) + cache_creation = usage.get("cache_creation_input_tokens") + cache_read = usage.get("cache_read_input_tokens") + if isinstance(cache_creation, (int, float)): + result["cached_input_tokens"] += int(cache_creation) + if isinstance(cache_read, (int, float)): + result["cached_input_tokens"] += int(cache_read) + if result["input_tokens"] == 0 and isinstance(usage.get("prompt_tokens"), (int, float)): + result["input_tokens"] = int(usage.get("prompt_tokens") or 0) + if result["output_tokens"] == 0 and isinstance(usage.get("completion_tokens"), (int, float)): + result["output_tokens"] = int(usage.get("completion_tokens") or 0) if result["total_tokens"] == 0: - result["total_tokens"] = result["input_tokens"] + result["output_tokens"] + result["total_tokens"] = result["input_tokens"] + result["cached_input_tokens"] + result["output_tokens"] return result @@ -301,12 +378,17 @@ def _infer_session_id(path: Path, record: SessionRecord) -> str: def _extract_request_features(text: str) -> Dict[str, str]: features: Dict[str, str] = {} patterns = { - "company": r"(?im)^\s*Company\s*:\s*(.+)$", - "project": r"(?im)^\s*Project\s*:\s*(.+)$", + "company": r"(?im)^\s*(?:Paperclip\s+Company|Company)\s*:\s*(.+)$", + "company_id": r"(?im)^\s*(?:Paperclip[- ]?Company[- ]?ID|Company\s+ID)\s*:\s*([0-9a-f-]{16,})\s*$", + "project": r"(?im)^\s*(?:Paperclip\s+Project|Project)\s*:\s*(.+)$", + "project_id": r"(?im)^\s*(?:Paperclip[- ]?Project[- ]?ID|Project\s+ID)\s*:\s*([0-9a-f-]{16,})\s*$", "queue": r"(?im)^\s*Queue\s*:\s*(.+)$", - "task": r"(?im)^\s*(?:Task|Title)\s*:\s*(.+)$", + "task": r"(?im)^\s*(?:Paperclip\s+Task|Task|Title)\s*:\s*(.+)$", "submitted_by": r"(?im)^\s*Submitted by\s*:\s*(.+)$", - "agent": r"(?im)^\s*Agent\s*:\s*(.+)$", + "agent": r"(?im)^\s*(?:Paperclip\s+Agent|Agent)\s*:\s*(.+)$", + "agent_id": r"(?im)^\s*(?:Paperclip[- ]?Agent[- ]?ID|Agent\s+ID|Staff\s+ID)\s*:\s*([0-9a-f-]{16,})\s*$", + "staff": r"(?im)^\s*(?:Paperclip\s+Staff|Staff|Role)\s*:\s*(.+)$", + "issue": r"(?im)^\s*(?:Paperclip\s+Issue|Issue|Issue\s+Key)\s*:\s*([A-Z]{2,10}-\d+).*$", } for key, pattern in patterns.items(): match = re.search(pattern, text) @@ -316,3 +398,10 @@ def _extract_request_features(text: str) -> Dict[str, str]: if issue: features["issue"] = issue.group(1) return features + + +def _read_json(path: Path) -> Any: + try: + return json.loads(path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + return None diff --git a/src/codex_usage_profiler/models.py b/src/codex_usage_profiler/models.py index e0a860d..14dbbf3 100644 --- a/src/codex_usage_profiler/models.py +++ b/src/codex_usage_profiler/models.py @@ -36,6 +36,9 @@ class Estimate: class SessionRecord: session_id: str path: str + source_path: Optional[str] = None + collector: Optional[str] = None + collector_tool: Optional[str] = None start_time: Optional[str] = None end_time: Optional[str] = None cwd: Optional[str] = None @@ -74,6 +77,9 @@ def to_dict(self, include_snippets: bool = False) -> Dict[str, Any]: data: Dict[str, Any] = { "session_id": self.session_id, "path": self.path, + "source_path": self.source_path, + "collector": self.collector, + "collector_tool": self.collector_tool, "start_time": self.start_time, "end_time": self.end_time, "cwd": self.cwd, diff --git a/src/codex_usage_profiler/paperclip.py b/src/codex_usage_profiler/paperclip.py index 50124df..7554fc3 100644 --- a/src/codex_usage_profiler/paperclip.py +++ b/src/codex_usage_profiler/paperclip.py @@ -1,6 +1,7 @@ from __future__ import annotations import re +import json from collections import Counter, defaultdict from dataclasses import dataclass, field from pathlib import Path @@ -29,32 +30,39 @@ class PaperclipIndex: companies: Dict[str, str] = field(default_factory=dict) agents: Dict[str, PaperclipAgent] = field(default_factory=dict) projects: Dict[str, str] = field(default_factory=dict) + company_votes: Dict[str, Counter[str]] = field(default_factory=lambda: defaultdict(Counter)) warnings: List[str] = field(default_factory=list) -def build_paperclip_index(config: Config) -> PaperclipIndex: +def build_paperclip_index(config: Config, paths: Optional[List[str]] = None) -> PaperclipIndex: root = Path(config.paperclip_root).expanduser() index = PaperclipIndex(root=str(root)) - if not config.paperclip_enabled or not root.exists(): + if not config.paperclip_enabled: return index - _index_agents(root, index) - _index_projects(root, index) + if root.exists(): + _index_agents(root, index) + _index_projects(root, index) + _index_collected_metadata(paths or [], index) + _finalize_company_labels(index) + _apply_config_aliases(index, config) return index def apply_paperclip_attribution(records: List[SessionRecord], index: PaperclipIndex) -> None: - if not index.companies and not index.agents and not index.projects: - return for record in records: context = _context_from_record(record) company_id = context.get("company_id") agent_id = context.get("agent_id") project_id = context.get("project_id") + if not agent_id and context.get("workspace_id") in index.agents: + agent_id = context.get("workspace_id") + agent = index.agents.get(agent_id or "") if agent_id else None + if not company_id and agent: + company_id = agent.company_id company_label = _feature(record, "company") or (index.companies.get(company_id or "") if company_id else None) project_label = _feature(record, "project") or (index.projects.get(project_id or "") if project_id else None) - agent = index.agents.get(agent_id or "") if agent_id else None - staff_label = _feature(record, "submitted_by") or _feature(record, "agent") or (agent.staff_label if agent else None) + staff_label = _feature(record, "staff") or _feature(record, "submitted_by") or _feature(record, "agent") or (agent.staff_label if agent else None) paperclip_task = _paperclip_task_label(record) if company_label: @@ -102,7 +110,7 @@ def _index_agents(root: Path, index: PaperclipIndex) -> None: staff = _extract_staff_label(text) or f"paperclip-agent:{agent_id[:8]}" company = _extract_company_label(text) if company: - index.companies[company_id] = _choose_company_label(index.companies.get(company_id), company) + _vote_company(index, company_id, company) index.agents[agent_id] = PaperclipAgent( company_id=company_id, agent_id=agent_id, @@ -114,7 +122,6 @@ def _index_agents(root: Path, index: PaperclipIndex) -> None: def _index_projects(root: Path, index: PaperclipIndex) -> None: project_votes: Dict[str, Counter[str]] = defaultdict(Counter) - company_votes: Dict[str, Counter[str]] = defaultdict(Counter) projects_root = root / "projects" if not projects_root.exists(): return @@ -127,18 +134,120 @@ def _index_projects(root: Path, index: PaperclipIndex) -> None: project_id = parts[parts.index("projects") + 2] except (ValueError, IndexError): continue - for candidate in _labels_from_project_files(path): + for candidate in _project_labels_from_project_files(path): project_votes[project_id][candidate] += 1 - company_votes[company_id][candidate] += 1 + for candidate in _company_labels_from_project_files(path): + _vote_company(index, company_id, candidate, weight=8) for project_id, votes in project_votes.items(): if votes: index.projects[project_id] = votes.most_common(1)[0][0] - for company_id, votes in company_votes.items(): - if votes and company_id not in index.companies: - index.companies[company_id] = votes.most_common(1)[0][0] -def _labels_from_project_files(path: Path) -> Iterable[str]: +def _index_collected_metadata(paths: List[str], index: PaperclipIndex) -> None: + for metadata_path in _discover_collected_metadata(paths): + try: + payload = json.loads(metadata_path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError) as exc: + index.warnings.append(f"paperclip_metadata_read_failed:{metadata_path}:{type(exc).__name__}") + continue + if not isinstance(payload, dict): + continue + _index_metadata_payload(payload, metadata_path, index) + + +def _discover_collected_metadata(paths: List[str]) -> List[Path]: + found: List[Path] = [] + seen: set[Path] = set() + for raw in paths: + path = Path(raw).expanduser() + candidates: Iterable[Path] + if path.is_file() and path.name == "paperclip-metadata.json": + candidates = [path] + elif path.is_dir(): + candidates = path.rglob("paperclip-metadata.json") + else: + candidates = [] + for candidate in candidates: + resolved = candidate.resolve() + if resolved not in seen: + seen.add(resolved) + found.append(resolved) + return found + + +def _index_metadata_payload(payload: Dict[str, object], path: Path, index: PaperclipIndex) -> None: + company_by_prefix: Dict[str, str] = {} + for row in payload.get("companies", []) if isinstance(payload.get("companies"), list) else []: + if not isinstance(row, dict): + continue + company_id = str(row.get("id") or "") + name = str(row.get("name") or "") + prefix = str(row.get("issuePrefix") or "").upper() + if company_id and name: + index.companies[company_id] = name + _vote_company(index, company_id, name, weight=20) + if company_id and prefix: + company_by_prefix[prefix] = company_id + for row in payload.get("projects", []) if isinstance(payload.get("projects"), list) else []: + if not isinstance(row, dict): + continue + project_id = str(row.get("id") or "") + name = str(row.get("name") or "") + company_id = str(row.get("companyId") or "") + if project_id and name: + index.projects[project_id] = name + if company_id and company_id in index.companies: + _vote_company(index, company_id, index.companies[company_id], weight=5) + for row in payload.get("agents", []) if isinstance(payload.get("agents"), list) else []: + if not isinstance(row, dict): + continue + agent_id = str(row.get("id") or "") + company_id = str(row.get("companyId") or "") + if not agent_id or not company_id: + continue + staff = _staff_label_from_metadata_agent(row) + index.agents[agent_id] = PaperclipAgent( + company_id=company_id, + agent_id=agent_id, + staff_label=staff, + company_label=index.companies.get(company_id), + evidence=[str(path)], + ) + for row in payload.get("workspaces", []) if isinstance(payload.get("workspaces"), list) else []: + if not isinstance(row, dict): + continue + workspace_id = str(row.get("id") or "") + company_id = str(row.get("companyId") or "") + if not company_id: + prefixes = [str(item).upper() for item in row.get("issuePrefixes", [])] if isinstance(row.get("issuePrefixes"), list) else [] + for prefix in prefixes: + if prefix in company_by_prefix: + company_id = company_by_prefix[prefix] + break + staff = str(row.get("staffLabel") or "") or f"paperclip-workspace:{workspace_id[:8]}" + if workspace_id and company_id: + index.agents.setdefault( + workspace_id, + PaperclipAgent( + company_id=company_id, + agent_id=workspace_id, + staff_label=staff, + company_label=index.companies.get(company_id), + evidence=[str(path)], + ), + ) + + +def _staff_label_from_metadata_agent(row: Dict[str, object]) -> str: + for key in ["title", "name", "role"]: + value = str(row.get(key) or "").strip() + if value: + return bounded_snippet(value, 80) + agent_id = str(row.get("id") or "") + return f"paperclip-agent:{agent_id[:8]}" + + +def _project_labels_from_project_files(path: Path) -> Iterable[str]: names = [item.name for item in path.iterdir() if item.is_file() and item.suffix.lower() in {".md", ".json", ".jsonl"}] blob = " ".join(names).lower() if "dinner-with-kids" in blob or "family-dinners" in blob: @@ -155,21 +264,48 @@ def _labels_from_project_files(path: Path) -> Iterable[str]: yield cleaned.title() +def _company_labels_from_project_files(path: Path) -> Iterable[str]: + names = [item.name.lower() for item in path.iterdir() if item.is_file()] + prefixes = Counter() + for name in names: + match = re.match(r"([a-z]+)-\d+[-_.]", name) + if match: + prefixes[match.group(1)] += 1 + if prefixes.get("max"): + yield "Maximum Goat" + if prefixes.get("min"): + yield "Min Apps" + if prefixes.get("dys"): + yield "DY Sphere" + if prefixes.get("sta"): + yield "STA" + + def _context_from_record(record: SessionRecord) -> Dict[str, str]: - blob = "\n".join(value for value in [record.path, record.cwd or "", *record.workspace_roots] if value) - result: Dict[str, str] = {} + blob = "\n".join(value for value in [record.path, record.source_path or "", record.cwd or "", *record.workspace_roots] if value) + result: Dict[str, str] = { + key: value + for key, value in { + "company_id": _feature(record, "company_id"), + "agent_id": _feature(record, "agent_id"), + "project_id": _feature(record, "project_id"), + "workspace_id": _feature(record, "workspace_id"), + }.items() + if value + } match = re.search(rf"/(?:\.paperclip|paperclip)/instances/[^/]+/companies/({UUID_RE})(?:/agents/({UUID_RE}))?/codex-home/", blob) if match: - result["company_id"] = match.group(1) + result.setdefault("company_id", match.group(1)) if match.group(2): - result["agent_id"] = match.group(2) + result.setdefault("agent_id", match.group(2)) match = re.search(rf"/(?:\.paperclip|paperclip)/instances/[^/]+/projects/({UUID_RE})/({UUID_RE})/", blob) if match: - result["company_id"] = match.group(1) - result["project_id"] = match.group(2) + result.setdefault("company_id", match.group(1)) + result.setdefault("project_id", match.group(2)) match = re.search(rf"/(?:\.paperclip|paperclip)/instances/[^/]+/workspaces/({UUID_RE})(?:/|$)", blob) if match: - result["agent_id"] = match.group(1) + result.setdefault("workspace_id", match.group(1)) + result.setdefault("agent_id", match.group(1)) return result @@ -207,14 +343,29 @@ def _extract_company_label(text: str) -> Optional[str]: return None -def _choose_company_label(current: Optional[str], candidate: str) -> str: - if not current: - return candidate - if current.startswith("paperclip-company:"): - return candidate - if len(candidate) < len(current) or candidate in {"DY Sphere", "STA", "Dinner with Kids", "Min Apps"}: - return candidate - return current +def _vote_company(index: PaperclipIndex, company_id: str, candidate: str, weight: int = 1) -> None: + if not company_id or not candidate: + return + index.company_votes[company_id][candidate] += weight + + +def _finalize_company_labels(index: PaperclipIndex) -> None: + for company_id, votes in index.company_votes.items(): + if votes: + index.companies[company_id] = votes.most_common(1)[0][0] + + +def _apply_config_aliases(index: PaperclipIndex, config: Config) -> None: + for company_id, label in config.paperclip_company_aliases.items(): + if label: + index.companies[company_id] = label + for project_id, label in config.paperclip_project_aliases.items(): + if label: + index.projects[project_id] = label + for agent_id, label in config.paperclip_agent_aliases.items(): + agent = index.agents.get(agent_id) + if agent and label: + agent.staff_label = label def _clean_company(value: str) -> str: @@ -235,10 +386,13 @@ def _paperclip_task_label(record: SessionRecord) -> Optional[str]: return features["issue"] if features.get("task"): return features["task"] - path_blob = record.cwd or "" - match = re.search(r"\b(DYS-\d+)\b", path_blob, re.IGNORECASE) + path_blob = "\n".join(value for value in [record.cwd or "", record.source_path or "", record.path, *record.workspace_roots] if value) + match = re.search(r"\b([A-Z]{2,10}-\d+)\b", path_blob, re.IGNORECASE) if match: return match.group(1).upper() + match = re.search(r"\b([a-z]{2,10})[-_](\d{1,6})\b", path_blob, re.IGNORECASE) + if match: + return f"{match.group(1).upper()}-{match.group(2)}" return None diff --git a/src/codex_usage_profiler/reporting.py b/src/codex_usage_profiler/reporting.py index a266b82..cd8d8d6 100644 --- a/src/codex_usage_profiler/reporting.py +++ b/src/codex_usage_profiler/reporting.py @@ -13,6 +13,8 @@ def render_json( telemetry: CodexBarTelemetry, reconciliation: List[Dict[str, Any]], warnings: List[str], + paperclip_spend: Optional[Dict[str, Any]] = None, + plan_analysis: Optional[Dict[str, Any]] = None, include_snippets: bool = False, ) -> str: payload = { @@ -23,6 +25,8 @@ def render_json( }, "telemetry": telemetry.to_dict(), "aggregates": aggregates, + "paperclip_spend": paperclip_spend or {}, + "plan_analysis": plan_analysis or {}, "findings": [item.to_dict() for item in findings], "reconciliation": reconciliation, "sessions": [record.to_dict(include_snippets=include_snippets) for record in records], @@ -38,6 +42,8 @@ def render_text( telemetry: CodexBarTelemetry, reconciliation: List[Dict[str, Any]], warnings: List[str], + paperclip_spend: Optional[Dict[str, Any]] = None, + plan_analysis: Optional[Dict[str, Any]] = None, top: int = 10, include_snippets: bool = False, ) -> str: @@ -51,9 +57,11 @@ def render_text( lines.append(f"Rate-card cost: ${total_cost:,.2f} (directional replacement-cost estimate)") lines.append("") lines.extend(_render_codexbar(telemetry)) + lines.extend(_render_plan_analysis(plan_analysis or {})) lines.extend(_render_aggregate("By client", aggregates.get("client", []), top)) lines.extend(_render_aggregate("By project", aggregates.get("project", []), top)) lines.extend(_render_aggregate("By task", aggregates.get("task", []), min(top, 15))) + lines.extend(_render_paperclip_company_spend(paperclip_spend or {}, top)) lines.extend(_render_aggregate("Paperclip companies", aggregates.get("paperclip_company", []), top)) lines.extend(_render_aggregate("Paperclip staff", aggregates.get("paperclip_staff", []), top)) lines.extend(_render_aggregate("Paperclip tasks", aggregates.get("paperclip_task", []), top)) @@ -120,6 +128,74 @@ def _render_aggregate(title: str, rows: List[Dict[str, Any]], top: int) -> List[ return lines +def _render_plan_analysis(plan: Dict[str, Any]) -> List[str]: + lines = ["Plan comparison", "---------------"] + if not plan: + return lines + ["none", ""] + lines.append( + f"Observed span: {int(plan.get('observed_span_days') or 1)}d | " + f"observed ${float(plan.get('observed_cost_usd') or 0.0):,.2f} | " + f"projected {int(plan.get('projection_days') or 30)}d ${float(plan.get('projected_cost_usd') or 0.0):,.2f}" + ) + rows = plan.get("plans") if isinstance(plan.get("plans"), list) else [] + if not rows: + lines.append("Add --monthly-plan-price-usd or --plan-price NAME=USD to compare plans.") + lines.append("") + return lines + lines.append(_fmt_row("plan", "price", "projected", "ratio", "delta")) + for row in rows[:8]: + ratio = row.get("projected_vs_price_percent") + lines.append( + _fmt_row( + str(row.get("plan", "plan"))[:34], + f"${float(row.get('monthly_price_usd') or 0.0):,.2f}", + f"${float(row.get('projected_rate_card_cost_usd') or 0.0):,.2f}", + "n/a" if ratio is None else f"{float(ratio):.0f}%", + f"${float(row.get('delta_usd') or 0.0):+,.2f}", + ) + ) + lines.append("") + return lines + + +def _render_paperclip_company_spend(spend: Dict[str, Any], top: int) -> List[str]: + lines = ["Paperclip company spend", "-----------------------"] + totals = spend.get("company_totals") if isinstance(spend.get("company_totals"), list) else [] + if not totals: + return lines + ["none", ""] + lines.append(_fmt_row("company", "sessions", "tokens", "cost", "proj30d")) + for row in totals[:top]: + lines.append( + _fmt_row( + str(row.get("company", "unknown"))[:34], + str(row.get("sessions", 0)), + f"{int(row.get('total_tokens', 0)):,}", + f"${float(row.get('estimated_cost_usd', 0.0)):,.2f}", + f"${float(row.get('projected_cost_usd', 0.0)):,.2f}", + ) + ) + daily = spend.get("daily") if isinstance(spend.get("daily"), list) else [] + if daily: + lines.append("") + lines.append("Latest Paperclip company days") + lines.append(_fmt_row("day/company", "sessions", "tokens", "cost", "staff")) + latest = sorted(daily, key=lambda row: str(row.get("period", "")), reverse=True)[:top] + for row in latest: + staff = row.get("top_staff") if isinstance(row.get("top_staff"), dict) else {} + top_staff = next(iter(staff.keys()), "unknown") + lines.append( + _fmt_row( + f"{row.get('period', 'unknown')} {row.get('company', 'unknown')}"[:34], + str(row.get("sessions", 0)), + f"{int(row.get('total_tokens', 0)):,}", + f"${float(row.get('estimated_cost_usd', 0.0)):,.2f}", + str(top_staff)[:12], + ) + ) + lines.append("") + return lines + + def _render_sessions(records: List[SessionRecord], top: int, include_snippets: bool) -> List[str]: lines = ["Top sessions", "------------"] sorted_records = sorted(records, key=lambda r: r.usage.get("total_tokens", 0), reverse=True) diff --git a/src/codex_usage_profiler/syslog_util.py b/src/codex_usage_profiler/syslog_util.py new file mode 100644 index 0000000..e07bd72 --- /dev/null +++ b/src/codex_usage_profiler/syslog_util.py @@ -0,0 +1,76 @@ +from __future__ import annotations + +import datetime as dt +import json +import socket +import time +from typing import Any, Dict, Optional +from urllib.error import URLError +from urllib.request import Request, urlopen + + +def emit_syslog_json( + event: Dict[str, Any], + *, + host: Optional[str] = None, + port: int = 514, + tag: str = "codex-usage-profiler", + protocol: str = "tcp", + timeout: float = 2.0, +) -> bool: + if not host: + return False + payload = json.dumps(event, sort_keys=True, separators=(",", ":")) + now = dt.datetime.now().strftime("%b %d %H:%M:%S") + hostname = socket.gethostname().split(".")[0] + message = f"<14>{now} {hostname} {tag}: {payload}\n".encode("utf-8", errors="replace") + sock_type = socket.SOCK_DGRAM if protocol == "udp" else socket.SOCK_STREAM + try: + with socket.socket(socket.AF_INET, sock_type) as sock: + sock.settimeout(timeout) + if sock_type == socket.SOCK_DGRAM: + sock.sendto(message, (host, port)) + else: + sock.connect((host, port)) + sock.sendall(message) + return True + except OSError: + return False + + +def emit_loki_json( + event: Dict[str, Any], + *, + url: Optional[str] = None, + labels: Optional[Dict[str, str]] = None, + timeout: float = 2.0, +) -> bool: + if not url: + return False + stream_labels = {"job": "codex-usage-profiler"} + stream_labels.update({str(k): str(v) for k, v in (labels or {}).items() if v is not None}) + payload = { + "streams": [ + { + "stream": stream_labels, + "values": [ + [ + str(time.time_ns()), + json.dumps(event, sort_keys=True, separators=(",", ":")), + ] + ], + } + ] + } + request = Request( + url, + data=json.dumps(payload).encode("utf-8"), + headers={"Content-Type": "application/json"}, + method="POST", + ) + try: + with urlopen(request, timeout=timeout) as response: + response.read() + return True + except (OSError, URLError): + return False diff --git a/tests/dashboard_e2e.mjs b/tests/dashboard_e2e.mjs index 8f41846..9490281 100644 --- a/tests/dashboard_e2e.mjs +++ b/tests/dashboard_e2e.mjs @@ -169,11 +169,14 @@ async function assertFlowGeometry(page, label) { }; }).filter((link) => link.sxDelta > 2 || link.txDelta > 2 || !link.syInside || !link.tyInside); const clippedNodes = nodes.filter((node) => node.y < 0 || node.y + node.height > flow.clientHeight + 1); - return { nodeCount: nodes.length, linkCount: document.querySelectorAll(".flow-link").length, stageGaps, badLinks, clippedNodes, flowHeight: flow.clientHeight }; + const horizontalClippedNodes = nodes.filter((node) => node.x < -1 || node.x + node.width > flow.clientWidth + 1); + return { nodeCount: nodes.length, linkCount: document.querySelectorAll(".flow-link").length, stageGaps, badLinks, clippedNodes, horizontalClippedNodes, flowHeight: flow.clientHeight, flowWidth: flow.clientWidth }; }); assert.equal(result.badLinks.length, 0, `${label}: bad Sankey link endpoints ${JSON.stringify(result.badLinks.slice(0, 3))}`); assert.equal(result.clippedNodes.length, 0, `${label}: clipped Sankey nodes ${JSON.stringify(result.clippedNodes)}`); - assert(result.stageGaps.every((gap) => gap >= 20), `${label}: stage gaps too small ${JSON.stringify(result.stageGaps)}`); + assert.equal(result.horizontalClippedNodes.length, 0, `${label}: horizontally clipped Sankey nodes ${JSON.stringify(result.horizontalClippedNodes)}`); + const minGap = result.flowWidth < 520 ? 2 : 20; + assert(result.stageGaps.every((gap) => gap >= minGap), `${label}: stage gaps too small ${JSON.stringify(result.stageGaps)}`); assert(result.nodeCount > 10, `${label}: expected many flow nodes`); assert(result.linkCount > 10, `${label}: expected many flow links`); } @@ -191,6 +194,52 @@ async function expectNoFilter(page, queryKey, label) { assert(!params.has(queryKey), `${label} did not clear ${queryKey}`); } +async function clickCardReset(page, card, queryKey, label) { + const button = page.locator(`.card-reset[data-reset-card="${card}"]`); + await assertEnabled(page, button, `${label} reset`); + await button.click(); + await expectNoFilter(page, queryKey, label); + await assertDisabled(page, button, `${label} reset`); +} + +async function assertEnabled(page, locator, label) { + await page.waitForTimeout(50); + assert(!(await locator.isDisabled()), `${label} was disabled`); +} + +async function assertDisabled(page, locator, label) { + await page.waitForTimeout(50); + assert(await locator.isDisabled(), `${label} was not disabled`); +} + +async function assertNoUnexpectedOverflow(page, label) { + const overflow = await page.evaluate(() => { + const allowed = [".table-wrap", ".company-day-bars", ".timeline"]; + return Array.from(document.querySelectorAll("body, .topbar, .brand, .dashboard, .panel, .panel-head, .heatmap, .coverage, .evidence-drawer")) + .filter((node) => !allowed.some((selector) => node.closest(selector))) + .map((node) => ({ selector: node.className || node.tagName, scrollWidth: node.scrollWidth, clientWidth: node.clientWidth })) + .filter((row) => row.scrollWidth > row.clientWidth + 2); + }); + assert.equal(overflow.length, 0, `${label}: unexpected horizontal overflow ${JSON.stringify(overflow.slice(0, 5))}`); +} + +async function waitForFlowSettled(page) { + // The app re-renders the flow behind a 60ms resize debounce, so a fixed + // sleep races the render under load. Wait until container width and node + // geometry have been stable for longer than the debounce window. + await page.waitForFunction(() => { + const flow = document.querySelector("#spend-flow"); + const nodes = Array.from(document.querySelectorAll(".flow-node")); + if (!flow || !nodes.length) return false; + const signature = `${flow.clientWidth}|${nodes.map((node) => node.getAttribute("style")).join(";")}`; + if (window.__flowSettle && window.__flowSettle.signature === signature) { + return performance.now() - window.__flowSettle.at > 150; + } + window.__flowSettle = { signature, at: performance.now() }; + return false; + }, { polling: 100, timeout: 15000 }); +} + async function run() { await mkdir(REPORTS, { recursive: true }); const work = await mkdtemp(path.join(tmpdir(), "cup-e2e-")); @@ -209,16 +258,21 @@ async function run() { page.on("pageerror", (err) => errors.push(err.message)); await page.goto(baseUrl); await page.waitForSelector(".flow-node"); + await waitForFlowSettled(page); await assertFlowGeometry(page, "initial"); + await assertNoUnexpectedOverflow(page, "initial 1440"); - for (const size of [{ width: 1120, height: 900 }, { width: 1728, height: 1117 }, { width: 1280, height: 900 }]) { + for (const size of [{ width: 1120, height: 900 }, { width: 1728, height: 1117 }, { width: 1280, height: 900 }, { width: 390, height: 860 }]) { await page.setViewportSize(size); - await page.waitForTimeout(100); + await waitForFlowSettled(page); await assertFlowGeometry(page, `viewport ${size.width}`); + await assertNoUnexpectedOverflow(page, `viewport ${size.width}`); } + await page.setViewportSize({ width: 1440, height: 1000 }); + await waitForFlowSettled(page); await page.locator("#hide-filters").click(); await page.locator("#close-drawer").click(); - await page.waitForTimeout(100); + await waitForFlowSettled(page); await assertFlowGeometry(page, "filters and drawer hidden"); const widenedFlow = await page.locator("#spend-flow").evaluate((node) => node.clientWidth); assert(widenedFlow > 500, `combined hidden layout collapsed flow to ${widenedFlow}px`); @@ -281,15 +335,13 @@ async function run() { const clientNode = '.flow-node[data-node-id="client|Codex Desktop"]'; await clickAndExpectFilter(page, clientNode, "clients"); assert.equal(await page.locator(clientNode).getAttribute("aria-pressed"), "true", "active flow node was not marked pressed"); - await page.locator(clientNode).click(); - await expectNoFilter(page, "clients", "second flow node click"); + await clickCardReset(page, "flow", "clients", "flow reset after node"); await page.locator("#reset-filters").click(); await page.waitForTimeout(100); await page.locator(".flow-link").first().click({ force: true }); assert(new URL(page.url()).searchParams.has("sessionIds"), "flow link did not filter"); assert(await page.locator(".flow-link.active").count(), "active flow link was not marked"); - await page.locator(".flow-link.active").first().click({ force: true }); - await expectNoFilter(page, "sessionIds", "second flow link click"); + await clickCardReset(page, "flow", "sessionIds", "flow reset after link"); await page.locator("#reset-filters").click(); await page.waitForTimeout(100); const otherNode = page.locator('.flow-node[data-node-id="client|Other / Unknown"]'); @@ -311,39 +363,47 @@ async function run() { target.dispatchEvent(new PointerEvent("pointerup", { bubbles: true, clientX: endX, clientY: y, pointerId: 1, pointerType: "mouse" })); }, { startX: brush.x + brush.width * 0.2, endX: brush.x + brush.width * 0.65, y: brush.y + brush.height / 2 }); assert(new URL(page.url()).searchParams.has("brushStartTime"), "brush drag did not set range"); - await page.locator("#reset-filters").click(); + await clickCardReset(page, "timeline", "brushStartTime", "timeline reset after brush"); await page.locator(".legend-useful").click(); assert(await page.locator(".legend-useful.inactive").count(), "legend did not toggle"); - await page.locator(".legend-useful").click(); + await assertEnabled(page, page.locator('.card-reset[data-reset-card="timeline"]'), "timeline reset after legend"); + await page.locator('.card-reset[data-reset-card="timeline"]').click(); + assert.equal(await page.locator(".legend-useful.inactive").count(), 0, "timeline reset did not restore legend"); await page.locator(".hour-bar").first().click(); assert(new URL(page.url()).searchParams.has("brushStartTime"), "timeline hour did not filter"); assert(await page.locator(".hour-bar.active").count(), "active timeline hour was not marked"); - await page.locator(".hour-bar.active").first().click(); - await expectNoFilter(page, "brushStartTime", "second timeline hour click"); + await clickCardReset(page, "timeline", "brushStartTime", "timeline reset after hour"); + + await page.locator(".company-spend-top button").first().click(); + assert(new URL(page.url()).searchParams.has("companies"), "company tile did not filter"); + await clickCardReset(page, "company", "companies", "company reset after tile"); + await page.locator(".company-day").first().click(); + assert(new URL(page.url()).searchParams.has("brushStartTime"), "company day did not filter"); + assert(await page.locator(".company-day.active").count(), "company day was not marked active"); + await clickCardReset(page, "company", "brushStartTime", "company reset after day"); + await page.locator(".heat-cell").nth(20).click(); assert(new URL(page.url()).searchParams.has("weekdays"), "heatmap cell did not filter"); assert(await page.locator(".heat-cell.active").count(), "active heatmap cell was not marked"); - await page.locator(".heat-cell.active").click(); - await expectNoFilter(page, "weekdays", "second heatmap cell click"); + await clickCardReset(page, "heatmap", "weekdays", "heatmap reset"); await page.locator("#reset-filters").click(); await page.locator(".driver-row").first().click(); assert(new URL(page.url()).searchParams.has("wasteKind"), "waste row did not filter"); assert(await page.locator(".driver-row.active").count(), "active waste row was not marked"); - await page.locator(".driver-row.active").click(); - await expectNoFilter(page, "wasteKind", "second waste row click"); + await clickCardReset(page, "waste", "wasteKind", "waste reset"); await page.locator("#reset-filters").click(); await page.locator(".coverage-list button").last().click(); assert(new URL(page.url()).searchParams.has("attributionCoverage"), "coverage row did not filter"); assert(await page.locator(".coverage-list button.active").count(), "active coverage row was not marked"); - await page.locator(".coverage-list button.active").click(); - await expectNoFilter(page, "attributionCoverage", "second coverage row click"); + await clickCardReset(page, "coverage", "attributionCoverage", "coverage reset"); await page.locator("#reset-filters").click(); await page.locator("#reduction-select").selectOption("75"); await page.locator(".projection-action").click(); assert.equal(await page.locator("#waste-filter").inputValue(), "any"); assert(await page.locator(".projection-action.active").count(), "projection action was not marked active"); - await page.locator(".projection-action.active").click(); + await assertEnabled(page, page.locator('.card-reset[data-reset-card="projection"]'), "projection reset"); + await page.locator('.card-reset[data-reset-card="projection"]').click(); assert.equal(await page.locator("#waste-filter").inputValue(), "all"); await page.locator("#reset-filters").click(); diff --git a/tests/test_collector.py b/tests/test_collector.py new file mode 100644 index 0000000..395dcb0 --- /dev/null +++ b/tests/test_collector.py @@ -0,0 +1,174 @@ +from __future__ import annotations + +import json +import tempfile +import unittest +from pathlib import Path +from unittest import mock + +from codex_usage_profiler.collector import discover_candidates, load_collector_config, run_collection, stage_file, stage_paperclip_metadata +from codex_usage_profiler.ingest import parse_logs +from codex_usage_profiler.attribution import attribute_sessions +from codex_usage_profiler.config import Config + + +class CollectorTests(unittest.TestCase): + def test_config_defaults_and_custom_globs(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + config_path = Path(tmp) / "collector.json" + config_path.write_text( + json.dumps( + { + "collector_name": "fixture-host", + "include_defaults": False, + "tools": {"kimi": str(Path(tmp) / "kimi" / "*.jsonl")}, + } + ), + encoding="utf-8", + ) + config = load_collector_config(str(config_path)) + self.assertEqual(config["collector_name"], "fixture-host") + self.assertEqual(list(config["tools"].keys()), ["kimi"]) + + def test_incremental_stage_push_and_metadata(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + source_dir = root / "source" / ".claude" / "projects" + source_dir.mkdir(parents=True) + session = source_dir / "session.jsonl" + session.write_text( + "\n".join( + [ + json.dumps({"timestamp": "2026-07-03T01:00:00Z", "cwd": "/work/demo", "model": "claude-test"}), + json.dumps({"timestamp": "2026-07-03T01:01:00Z", "usage": {"input_tokens": 10, "output_tokens": 5}}), + ] + ) + + "\n", + encoding="utf-8", + ) + config = { + "collector_name": "fixture-host", + "destination": str(root / "central"), + "include_defaults": False, + "tools": {"claude": [str(source_dir / "*.jsonl")]}, + "exclude": [], + "state_path": str(root / "state.json"), + "staging_dir": str(root / "staging"), + "log_path": str(root / "collector.log"), + "syslog_host": "", + "loki_url": "", + "max_file_bytes": 1_000_000, + } + first = run_collection(config) + self.assertEqual(first["matched_files"], 1) + self.assertEqual(first["staged_files"], 1) + self.assertEqual(first["pushed_files"], 1) + copied = list((root / "central" / "fixture-host" / "claude").glob("*/*.jsonl")) + self.assertEqual(len(copied), 1) + meta = json.loads(copied[0].with_suffix(".jsonl.meta.json").read_text(encoding="utf-8")) + self.assertEqual(meta["tool"], "claude") + self.assertEqual(meta["collector"], "fixture-host") + second = run_collection(config) + self.assertEqual(second["matched_files"], 1) + self.assertEqual(second["staged_files"], 0) + self.assertEqual(second["skipped_files"], 1) + + def test_collected_path_attributes_client(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / "collected-sessions" / "fixture-host" / "claude" / "abc" / "session.jsonl" + path.parent.mkdir(parents=True) + path.write_text( + json.dumps({"timestamp": "2026-07-03T01:00:00Z", "usage": {"input_tokens": 1, "output_tokens": 2}}) + "\n", + encoding="utf-8", + ) + records, warnings = parse_logs([str(path)]) + self.assertEqual(warnings, []) + attribute_sessions(records, Config()) + self.assertEqual(records[0].client.label, "Claude Code") + self.assertEqual(records[0].project.label, "fixture-host:claude") + self.assertEqual(records[0].usage["total_tokens"], 3) + + def test_discover_skips_excludes_and_oversized_candidates_are_counted(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + good = root / "good.jsonl" + auth = root / "auth.json" + good.write_text("{}\n", encoding="utf-8") + auth.write_text("{}\n", encoding="utf-8") + codexbar = root / "Library" / "Caches" / "CodexBar" / "cost-usage" / "codex-v8.json" + codexbar.parent.mkdir(parents=True) + codexbar.write_text("{}\n", encoding="utf-8") + config = { + "include_defaults": False, + "tools": {"codex": [str(root / "*.json*")], "codexbar": [str(codexbar)]}, + "exclude": ["**/auth.json"], + } + self.assertEqual(discover_candidates(config), [("codex", good.resolve()), ("codexbar", codexbar.resolve())]) + + def test_missing_source_during_stage_is_not_fatal(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + source = root / "source.jsonl" + source.write_text("{}\n", encoding="utf-8") + config = { + "collector_name": "fixture-host", + "destination": str(root / "central"), + "include_defaults": False, + "tools": {"codex": [str(source)]}, + "exclude": [], + "state_path": str(root / "state.json"), + "staging_dir": str(root / "staging"), + "log_path": str(root / "collector.log"), + "syslog_host": "", + "loki_url": "", + } + with mock.patch("codex_usage_profiler.collector.stage_file", side_effect=FileNotFoundError): + result = run_collection(config) + self.assertEqual(result["errors"], []) + self.assertEqual(result["vanished_files"], 1) + + def test_stage_file_adds_structured_paperclip_sidecar_context(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + source = root / ".paperclip" / "instances" / "default" / "companies" / "67a988b1-d14e-4f38-8d77-ca55848888b0" / "agents" / "08668359-96b5-4049-896f-f11e9925f771" / "codex-home" / "sessions" / "session.jsonl" + source.parent.mkdir(parents=True) + source.write_text("{}\n", encoding="utf-8") + copied = stage_file(root / "staging", "fixture-host", "codex", source, source.stat()) + meta = json.loads(copied.with_suffix(".jsonl.meta.json").read_text(encoding="utf-8")) + self.assertEqual(meta["paperclip"]["company_id"], "67a988b1-d14e-4f38-8d77-ca55848888b0") + self.assertEqual(meta["paperclip"]["agent_id"], "08668359-96b5-4049-896f-f11e9925f771") + + def test_stage_paperclip_metadata_snapshot(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + instance = root / ".paperclip" / "instances" / "default" + workspace = instance / "workspaces" / "8ec73631-7c0f-4049-8a96-c205536c3cce" + workspace.mkdir(parents=True) + (workspace / "sre-sta-42-heartbeat.md").write_text("ok", encoding="utf-8") + companies = [{"id": "67a988b1-d14e-4f38-8d77-ca55848888b0", "name": "Start2Scale", "issuePrefix": "STA", "status": "active"}] + agents = [{"id": "08668359-96b5-4049-896f-f11e9925f771", "companyId": companies[0]["id"], "name": "Security Engineer", "title": "Security Engineer"}] + projects = [{"id": "64b67709-14d6-4158-bb48-fec54144e061", "companyId": companies[0]["id"], "name": "Start2Scale"}] + + def fake_fetch(url, timeout, warnings, label): + if label == "companies": + return companies + if label.startswith("agents:"): + return agents + if label.startswith("projects:"): + return projects + return [] + + with mock.patch("codex_usage_profiler.collector._fetch_json_list", side_effect=fake_fetch): + result = stage_paperclip_metadata(root / "staging", "fixture-host", {"paperclip_root": str(instance)}) + self.assertIsNotNone(result) + snapshots = list((root / "staging").glob("fixture-host/paperclip_metadata/*/paperclip-metadata.json")) + self.assertEqual(len(snapshots), 1) + payload = json.loads(snapshots[0].read_text(encoding="utf-8")) + self.assertEqual(payload["companies"][0]["name"], "Start2Scale") + self.assertEqual(payload["agents"][0]["title"], "Security Engineer") + self.assertEqual(payload["workspaces"][0]["staffLabel"], "SRE") + self.assertEqual(payload["workspaces"][0]["companyId"], companies[0]["id"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_dashboard.py b/tests/test_dashboard.py index aa8b0e7..2fba636 100644 --- a/tests/test_dashboard.py +++ b/tests/test_dashboard.py @@ -89,6 +89,8 @@ def session( "run": {"session_count": len(sessions), "warning_count": 0, "confidence_note": "fixture"}, "telemetry": {"available": True, "live_usage": {"windows": [{"usedPercent": 40}]}, "warnings": []}, "aggregates": {}, + "paperclip_spend": {"projection_days": 30}, + "plan_analysis": {"projection_days": 30, "monthly_plan_price_usd": 20.0}, "findings": [ { "kind": "retry_loop", @@ -131,6 +133,24 @@ def test_dashboard_server_serves_static_assets_and_report(self) -> None: self.assertEqual(api["run"]["session_count"], 3) self.assertEqual(health, "ok\n") + def test_dashboard_server_accepts_syslog_options(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + report_path = Path(tmp) / "report.json" + report_path.write_text(json.dumps(sample_report()), encoding="utf-8") + server = make_server( + report_path, + "127.0.0.1", + 0, + syslog_host=None, + syslog_port=514, + loki_url="http://loki.example/api/v1/push", + ) + try: + self.assertEqual(server.RequestHandlerClass.syslog_port, 514) + self.assertEqual(server.RequestHandlerClass.loki_url, "http://loki.example/api/v1/push") + finally: + server.server_close() + def test_dashboard_data_functions_filter_sort_export_and_permalink(self) -> None: script = f""" const assert = require('assert'); @@ -223,6 +243,11 @@ def test_dashboard_data_functions_filter_sort_export_and_permalink(self) -> None const costBuckets = dashboard.hourlyBuckets(report.sessions, report, 'cost'); assert.strictEqual(Math.round(costBuckets.reduce((sum, bucket) => sum + bucket.total, 0) * 10) / 10, 6.5); +const companySpend = dashboard.companySpendModel(report.sessions, report, 'cost'); +assert.strictEqual(companySpend.totals[0].company, 'Acme'); +assert.strictEqual(companySpend.totals[0].sessions, 2); +assert(companySpend.totals[0].projectedCost > 0); +assert(companySpend.days.some((row) => row.day === '2026-06-30')); const coverage = dashboard.coverageStats(report.sessions); assert(coverage.unknownStaff > 10 && coverage.unknownStaff < 20); @@ -254,6 +279,8 @@ def test_dashboard_data_functions_filter_sort_export_and_permalink(self) -> None """ env = dict(os.environ) env["CUP_REPORT"] = json.dumps(sample_report()) + # Hour/day filters work in local time; expectations assume UTC+10. + env["TZ"] = "Australia/Sydney" proc = subprocess.run(["node", "-e", script], text=True, capture_output=True, env=env, check=False) self.assertEqual(proc.returncode, 0, proc.stderr) diff --git a/tests/test_profiler.py b/tests/test_profiler.py index 4ee32e0..a6b80e0 100644 --- a/tests/test_profiler.py +++ b/tests/test_profiler.py @@ -2,15 +2,24 @@ import json import os +import shutil import subprocess import sys import tempfile import unittest from pathlib import Path -from codex_usage_profiler.analysis import build_aggregates, classify_outcomes, find_waste_candidates, reconcile_codexbar +from codex_usage_profiler.analysis import ( + build_aggregates, + build_paperclip_spend, + build_plan_analysis, + classify_outcomes, + find_waste_candidates, + reconcile_codexbar, +) from codex_usage_profiler.attribution import attribute_sessions from codex_usage_profiler.codexbar import ( + collect_codexbar, _normalize_cost_output, _normalize_usage_output, _read_cost_cache, @@ -73,6 +82,25 @@ def test_codexbar_parsers(self) -> None: self.assertEqual(history["windows"][0]["latest"]["usedPercent"], 15) self.assertNotIn("secret", history["preferredAccountHash"]) + def test_codexbar_collected_root(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) / "collected-sessions" / "macbook-pro" / "codexbar" / "abc" + root.mkdir(parents=True) + shutil.copy2(FIXTURES / "codex-v8.json", root / "codex-v8.json") + shutil.copy2(FIXTURES / "models-dev-v1.json", root / "models-dev-v1.json") + shutil.copy2(FIXTURES / "history_codex.json", root / "codex.json") + env = dict(os.environ) + os.environ["CODEXBAR_COLLECTED_ROOT"] = str(Path(tmp) / "collected-sessions") + try: + telemetry = collect_codexbar(enabled=True, timeout=1) + finally: + os.environ.clear() + os.environ.update(env) + self.assertTrue(telemetry.available) + self.assertIsNotNone(telemetry.cost_cache) + self.assertIsNotNone(telemetry.pricing_cache) + self.assertIsNotNone(telemetry.history) + def test_estimates_and_reports(self) -> None: records, _ = parse_logs([str(FIXTURES / "complete.jsonl"), str(FIXTURES / "object_meta.jsonl")]) attribute_sessions(records, Config()) @@ -81,13 +109,18 @@ class Telemetry: apply_estimates(records, Config(), Telemetry()) classify_outcomes(records) aggregates = build_aggregates(records) + paperclip_spend = build_paperclip_spend(records, Config()) + plan_analysis = build_plan_analysis(records, Config(monthly_plan_price_usd=20.0)) findings = find_waste_candidates(records) cost = _normalize_cost_output(json.loads((FIXTURES / "codexbar_cost.json").read_text())) reconciliation = reconcile_codexbar(records, cost) self.assertGreater(aggregates["project"][0]["total_tokens"], 0) + self.assertIn("projected_cost_usd", plan_analysis) + self.assertEqual(paperclip_spend["company_totals"], []) self.assertTrue(any(r.estimate.confidence == "rate_card_estimate" for r in records)) - payload = json.loads(render_json(records, aggregates, findings, TelemetryLike(), reconciliation, [], False)) + payload = json.loads(render_json(records, aggregates, findings, TelemetryLike(), reconciliation, [], paperclip_spend, plan_analysis, False)) self.assertIn("sessions", payload) + self.assertIn("plan_analysis", payload) self.assertNotIn("first_request_snippet", payload["sessions"][0]) def test_paperclip_staff_company_project_attribution(self) -> None: @@ -98,12 +131,86 @@ def test_paperclip_staff_company_project_attribution(self) -> None: self.assertEqual(warnings, []) attribute_sessions(records, config) apply_paperclip_attribution(records, build_paperclip_index(config)) + class Telemetry: + pricing_cache = None + apply_estimates(records, config, Telemetry()) + classify_outcomes(records) record = records[0] self.assertEqual(record.paperclip_company.label, "STA") self.assertEqual(record.paperclip_staff.label, "Security Engineer") self.assertEqual(record.paperclip_project.label, "Start2Scale") self.assertEqual(record.paperclip_task.label, "STA-42") self.assertEqual(record.project.label, "Start2Scale") + spend = build_paperclip_spend(records, Config(paperclip_root=str(root), projection_days=30)) + self.assertEqual(spend["company_totals"][0]["company"], "STA") + self.assertEqual(spend["daily"][0]["period"], "2026-06-30") + self.assertGreater(spend["company_totals"][0]["projected_cost_usd"], 0) + + def test_collected_paperclip_sidecar_preserves_company_and_staff(self) -> None: + root = FIXTURES / "paperclip" / "instances" / "default" + source = root / "companies" / "67a988b1-d14e-4f38-8d77-ca55848888b0" / "agents" / "08668359-96b5-4049-896f-f11e9925f771" / "codex-home" / "sessions" / "2026" / "06" / "30" / "paperclip-agent.jsonl" + with tempfile.TemporaryDirectory() as tmp: + collected = Path(tmp) / "collected-sessions" / "macbook-pro" / "codex" / "abc" / "paperclip-agent.jsonl" + collected.parent.mkdir(parents=True) + shutil.copy2(source, collected) + collected.with_suffix(".jsonl.meta.json").write_text( + json.dumps({"collector": "macbook-pro", "tool": "codex", "source_path": str(source)}), + encoding="utf-8", + ) + config = Config(paperclip_root=str(root)) + records, warnings = parse_logs([str(collected)]) + self.assertEqual(warnings, []) + attribute_sessions(records, config) + apply_paperclip_attribution(records, build_paperclip_index(config)) + record = records[0] + self.assertEqual(record.source_path, str(source)) + self.assertEqual(record.paperclip_company.label, "STA") + self.assertEqual(record.paperclip_staff.label, "Security Engineer") + self.assertEqual(record.project.label, "Start2Scale") + + def test_paperclip_prompt_preamble_attribution(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + session = Path(tmp) / "session.jsonl" + text = "\n".join( + [ + "Paperclip Company ID: 67a988b1-d14e-4f38-8d77-ca55848888b0", + "Paperclip Company: Start2Scale", + "Paperclip Agent ID: 08668359-96b5-4049-896f-f11e9925f771", + "Paperclip Staff: Security Engineer", + "Paperclip Project ID: 64b67709-14d6-4158-bb48-fec54144e061", + "Paperclip Project: Start2Scale", + "Paperclip Issue: STA-42", + "Paperclip Task: Security review", + ] + ) + session.write_text( + json.dumps({"timestamp": "2026-06-30T10:00:00Z", "type": "message", "payload": {"message": {"role": "user", "content": text}}, "usage": {"input_tokens": 1, "output_tokens": 2}}) + "\n", + encoding="utf-8", + ) + records, _ = parse_logs([str(session)]) + attribute_sessions(records, Config()) + apply_paperclip_attribution(records, build_paperclip_index(Config(paperclip_root="/missing"))) + self.assertEqual(records[0].paperclip_company.label, "Start2Scale") + self.assertEqual(records[0].paperclip_staff.label, "Security Engineer") + self.assertEqual(records[0].paperclip_project.label, "Start2Scale") + self.assertEqual(records[0].paperclip_task.label, "STA-42") + + def test_paperclip_company_prefix_aliases(self) -> None: + root = FIXTURES / "paperclip" / "instances" / "default" + with tempfile.TemporaryDirectory() as tmp: + tmp_root = Path(tmp) / "instance" + project = tmp_root / "projects" / "d8126350-acdb-495b-ab18-469660d616d2" / "f278a9b4-0d9f-4fe9-88cb-bd20ad8bfd83" / "_default" + project.mkdir(parents=True) + (project / "max-17-venue-discovery-token-usage-audit.md").write_text("fixture", encoding="utf-8") + index = build_paperclip_index(Config(paperclip_root=str(tmp_root))) + self.assertEqual(index.companies["d8126350-acdb-495b-ab18-469660d616d2"], "Maximum Goat") + aliased = build_paperclip_index( + Config( + paperclip_root=str(root), + paperclip_company_aliases={"67a988b1-d14e-4f38-8d77-ca55848888b0": "Starter Scale"}, + ) + ) + self.assertEqual(aliased.companies["67a988b1-d14e-4f38-8d77-ca55848888b0"], "Starter Scale") def test_cli_e2e_paperclip_json_report(self) -> None: root = FIXTURES / "paperclip" / "instances" / "default"