Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ Common reports:
codex-usage-profiler --days 7 --top 20
codex-usage-profiler --since 2026-06-24
codex-usage-profiler --format json --output reports/latest.json
codex-usage-profiler --days 14 --monthly-plan-price-usd 200 --plan-price Plus=20 --plan-price Pro=200
codex-usage-dashboard --report reports/latest.json
```

Expand All @@ -65,9 +66,30 @@ codex-usage-profiler --no-codexbar --paths ~/.codex/sessions

Raw prompt and response bodies are not printed by default. Use `--include-snippets` only when you explicitly want bounded redacted task snippets in JSON/text output.

## Network Collection

For multi-machine setups, install the collector on each host and run the dashboard/report refresh on one always-on machine:

```bash
codex-usage-collector --print-default-config > ~/.config/codex-usage-profiler/collector.json
codex-usage-collector --config ~/.config/codex-usage-profiler/collector.json --once
```

The collector scans configurable JSON/JSONL session globs for Codex, Claude, Pi, T3Chat, Kimi, droid, Cursor, OpenCode, and Paperclip, stages changed files with metadata sidecars, and pushes them to the configured central destination. Linux systemd user units, macOS launchd plist, LXSO report refresh, dashboard service, and a rootless high-port `.lan` proxy are under `ops/`.

Central LXSO pattern:

```bash
ops/bin/codex-usage-run-report.sh
codex-usage-dashboard --report ~/.local/state/codex-usage-profiler/latest-report.json --host 0.0.0.0 --port 8775
```

Collector, report-refresh, and dashboard access events are written as JSONL and can also be forwarded to Orange Pi syslog and LXSO2 Loki. In Grafana, query Loki with `{job="codex-usage-profiler"}` and narrow by `service="collector"`, `service="report"`, or `service="dashboard"`.

## What The Tool Produces

- Usage grouped by client, project, task, model, day, hour, Paperclip company, Paperclip staff, and Paperclip task.
- Paperclip company spend by day/month, company 30-day projections, and configurable plan-price comparisons.
- Top sessions by observed tokens and rate-card cost.
- Current CodexBar quota windows when CodexBar is installed.
- CodexBar local-vs-cache ratio rows to reveal scope mismatch.
Expand All @@ -79,6 +101,7 @@ Raw prompt and response bodies are not printed by default. Use `--include-snippe
- `Tokens`: observed tokens found in local session logs.
- `Rate-card cost`: directional replacement-cost estimate from known model rates or CodexBar pricing caches.
- `Live quota now`: current CodexBar quota window, not a historical usage allocation.
- `Plan comparison`: projected local rate-card replacement cost vs configured plan price; useful for value/scale decisions, not official quota reconstruction.
- `Observed share`: share of tokens within the scanned local logs.
- `Durable output`: sessions with observable output signals such as edits, tests, commits, PRs, or action tools. This does not prove end-user value.
- `Review candidates`: sessions matching patterns worth inspecting. These can overlap with durable-output sessions.
Expand All @@ -105,6 +128,7 @@ This project was built with OpenSpec as an explicit design trail:

- `openspec/changes/build-usage-profiler/`: ingestion, attribution, quota/cost estimates, outcome evidence, reporting.
- `openspec/changes/add-session-dashboard-ui/`: compact dashboard, user stories, mockup validation, interaction contract.
- `openspec/changes/add-network-session-collection/`: multi-host collection, LXSO deployment, `.lan` access, and Grafana-visible logging.
- `docs/dashboard-final-interaction-contract.md`: card-by-card interaction expectations.
- `docs/critical-review.md`: subagent critique summary and which critical fixes were accepted.

Expand Down
4 changes: 4 additions & 0 deletions openspec/changes/add-session-dashboard-ui/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,10 @@ Use CSS bars and inline SVG for timeline, heatmap, rankings, coverage, and spend

Alternative: Chart.js/D3. Rejected for dependency weight.

### Card-local resets and responsive relayout

Each filterable visualization exposes a local reset button in the panel header. Active chart items still toggle off directly, but reset buttons make recovery obvious when the active item is clipped, scrolled away, or visually subtle. The Sankey/alluvial flow re-renders on viewport changes because its SVG links and DOM nodes are positioned from the container width.

### Privacy-safe evidence drawer

Show structured evidence such as edits, tests, commits, PRs, commands, token counts, attribution confidence, session IDs, and local paths. Do not show raw prompts unless a report explicitly includes snippets.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,12 @@ The dashboard SHALL visualize usage through an hourly timeline, day/hour heatmap
- **AND WHEN** the user clicks the active item again
- **THEN** the dashboard clears only that item's filter and returns the affected panel toward its default scope without clearing unrelated filters

#### Scenario: Card-local reset
- **WHEN** a visualization panel applies or reflects a local selection such as a flow selection, time brush, company/day selection, heatmap cell, review candidate, coverage bucket, or projection filter
- **THEN** that panel shows an enabled reset control
- **AND WHEN** the user activates the reset control
- **THEN** the dashboard clears that panel's local selection, updates the permalink, and disables the reset control again

### Requirement: Session Table And Evidence Drawer
The dashboard SHALL provide a sortable compact session table and a session evidence drawer showing attribution, token/rate-card cost estimates, outcome, review pattern, edits, tests, command labels, and linked identifiers when available.

Expand All @@ -76,6 +82,11 @@ The dashboard SHALL provide keyboard-accessible controls, visible focus states,
- **WHEN** the user navigates the page with the keyboard
- **THEN** interactive controls, table rows, exports, and the evidence drawer are reachable and labelled

#### Scenario: No unintended horizontal overflow
- **WHEN** the dashboard is viewed with both sidebars open at desktop widths or on a narrow mobile viewport
- **THEN** top-level regions, panel headers, heatmap, coverage, and flow panels fit within the viewport
- **AND** only deliberate scroll containers such as the session table may scroll horizontally

### Requirement: Mockup Validation
The implementation SHALL preserve the core structure of the final imagegen mockup: filter rail, KPI strip, main graph grid, session table, and evidence drawer.

Expand Down
8 changes: 8 additions & 0 deletions openspec/changes/add-session-dashboard-ui/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,11 @@
- [x] 4.2 Add data-function tests for filtering, sorting, summaries, CSV, and permalink state
- [x] 4.3 Run the unit and integration test suite
- [x] 4.4 Start the dashboard against the real report and verify local/Tailscale access details

## 5. Interaction And Layout Hardening

- [x] 5.1 Add card-local reset controls for all filterable visualization panels
- [x] 5.2 Make company day selections visibly active and reversible
- [x] 5.3 Re-render the Sankey flow after viewport/sidebar size changes
- [x] 5.4 Fix desktop/sidebar and mobile text overflow in panel headers, heatmap, coverage, and topbar
- [x] 5.5 Extend E2E tests for card resets, flow bounds, and viewport overflow
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-03
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
## Context

The profiler currently scans local paths. Alex's real usage spans several machines and harnesses: this Mac, LXSO1, LXSO2, LXSO3, NurseDroid, Codex, Claude, Pi, T3Chat, Kimi, droid, Cursor, OpenCode, and Paperclip. LXSO1 is the best central app host because it is always on and already runs agent services. `grafana.lan` and Loki now live on LXSO2, while Orange Pi still owns `.lan` DNS/Caddy and the syslog/Postgres observability path.

## Goals / Non-Goals

**Goals:**

- Run Codex Usage Profiler continuously on an LXSO machine.
- Collect session files from all reachable hosts without requiring tool-specific daemons.
- Make new tools easy to add with config globs.
- Preserve source host/tool/path metadata.
- Refresh the central report on a timer and serve the dashboard over LAN/Tailscale.
- Emit collector, report-refresh, and dashboard events to local JSONL logs, Orange Pi syslog/Postgres, and LXSO2 Loki for Grafana.

**Non-Goals:**

- Exact billing reconstruction.
- Parsing every proprietary chat database format in the collector.
- Replacing existing Grafana/Loki infrastructure.
- Copying auth files, caches, model downloads, browser profiles, or arbitrary app data.

## Decisions

1. Use push collectors over SSH/rsync.

Rationale: every machine can run a user-level timer and push changed files to LXSO1. No central credentials or inbound collector API are needed.

2. Use config-driven globs with conservative defaults.

Rationale: Codex and Claude have known JSONL locations, but T3Chat, Kimi, and droid session paths can vary by install. Defaults catch common session directories; per-host config can add or remove paths.

3. Stage files under `collector/tool/hash/basename` plus a metadata sidecar.

Rationale: this avoids path collisions, keeps source metadata, and lets the profiler infer client/host from central paths while source files stay untouched.

3a. Snapshot lightweight Paperclip topology.

Rationale: central LXSO reports do not have direct access to the Mac's Paperclip API or local company/agent instruction tree. The collector can safely stage a small `paperclip-metadata.json` snapshot with company, agent, project, and workspace labels. This avoids opaque UUID-only reports while keeping raw prompts and source data out of the snapshot.

4. Keep the collector dependency-free.

Rationale: LXSO, NurseDroid, Orange Pi, and macOS all have Python 3. A standard-library collector is easier to install than a daemon with a dependency tree.

5. Log to JSONL, syslog, and Loki.

Rationale: JSONL gives local troubleshooting. Syslog matches the existing Orange Pi ingestion path. Direct Loki push makes the same events visible in canonical `grafana.lan` without requiring journald scraping on every host.

6. Prefer LXSO1 for the dashboard; use a high-port `.lan` proxy if Caddy cannot be modified.

Rationale: Orange Pi Caddy is root-owned. When passwordless sudo is unavailable, `codex-usage.lan:8775` can still work through wildcard `.lan` DNS plus a user-level proxy.

## Risks / Trade-offs

- Tool paths can be wrong or incomplete -> keep config visible and log zero-match tools.
- First collection can copy many files -> skip oversized files and copy only mtime/size changes after the first run.
- Non-Codex logs can lack token fields -> central report labels token-known coverage rather than inventing numbers.
- Remote forwarding can fail independently -> local JSONL logs remain authoritative and forwarding failures do not fail collection.
- `.lan` without a port needs root Caddy changes -> provide a working high-port path and document the root-owned route.

## Migration Plan

1. Add collector, tests, and ops assets.
2. Deploy the repo to LXSO1 and install dashboard/report systemd user units.
3. Install collector configs/timers on this Mac, LXSO1, LXSO2, LXSO3, and NurseDroid.
4. Start a user-level Orange Pi LAN proxy if Caddy cannot be patched.
5. Run collectors once, refresh the report, check dashboard health, and query Loki/Grafana labels.
6. Roll back by disabling the user timers/services and deleting the central collection directory.

## Open Questions

- Whether to add direct parsers for T3Chat/Kimi/droid browser storage formats after real files are observed.
- Whether LXSO2 Loki should later scrape host journald directly in addition to direct push events.
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
## Why

Codex Usage Profiler is useful on one machine, but Alex's agent sessions are spread across the Mac, LXSO hosts, NurseDroid, and tool-specific homes. A network collector lets the profiler answer "what burned quota overnight?" from one LXSO-hosted dashboard instead of requiring ad hoc rsyncs and local-only scans.

## What Changes

- Add a portable session collector that scans configurable tool globs for Codex, Claude, Pi, T3Chat, Kimi, droid, Cursor, OpenCode, and Paperclip session files.
- Stage changed files with host/tool metadata and push them to a central LXSO collection directory.
- Add central LXSO report refresh and dashboard service templates.
- Add JSONL/syslog/Loki logging for collectors, report refreshes, and dashboard access events so activity appears in Grafana.
- Add deployment docs and service assets for Linux systemd user timers, macOS launchd, and the home `.lan` access path.

## Capabilities

### New Capabilities

- `network-session-collection`: Collect, centralize, profile, serve, and log multi-host AI coding session files.

### Modified Capabilities

None.

## Impact

- Adds a standard-library Python collector CLI and tests.
- Adds optional dashboard syslog and Loki forwarding.
- Adds ops files under `ops/` for systemd, launchd, report refresh, and LAN proxy deployment.
- Reads local user-level session files and copies them to the configured central host; it does not mutate source session stores.
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
## ADDED Requirements

### Requirement: Configurable Multi-Tool Collection
The system SHALL provide a collector that scans configured session-file globs for known AI coding/chat tools and skips missing paths without failing the run.

#### Scenario: Missing tool paths
- **WHEN** a host has no Kimi or T3Chat session directory
- **THEN** the collector completes successfully and logs zero matches for those tools

#### Scenario: Added custom path
- **WHEN** a collector config adds a new glob for a tool
- **THEN** matching files are included in the next collection run

### Requirement: Incremental Central Push
The system SHALL copy changed session files into a central collection directory with source host, tool, and original path metadata.

#### Scenario: Changed file copied
- **WHEN** a matched session file has a new size or modification time
- **THEN** the collector stages the file, writes a metadata sidecar, and pushes it to the configured destination

#### Scenario: Unchanged file skipped
- **WHEN** a matched session file is unchanged from collector state
- **THEN** the collector does not restage that file

#### Scenario: Paperclip source path attribution
- **WHEN** a collected session comes from a Paperclip company, agent, project, or workspace path
- **THEN** the metadata sidecar preserves the original `source_path`
- **AND** includes structured Paperclip identifiers when they can be parsed from the path

### Requirement: Paperclip Metadata Snapshot
The system SHALL optionally collect lightweight Paperclip topology metadata so the central report can map IDs to company, staff, project, and workspace names without direct access to the source machine's Paperclip API.

#### Scenario: Collector snapshots Paperclip topology
- **WHEN** the collector can reach the local Paperclip API or workspace directory
- **THEN** it stages a `paperclip-metadata.json` snapshot containing company IDs/names, issue prefixes, agent IDs/titles, project IDs/names, and workspace hints

#### Scenario: Central report consumes snapshot
- **WHEN** the central profiler scans collected sessions and metadata snapshots
- **THEN** Paperclip sessions are attributed by explicit metadata, original source path, agent/project indexes, and workspace hints in that precedence order

### Requirement: LXSO Dashboard Deployment
The system SHALL include service assets that run report refresh and dashboard serving on an LXSO host using the central collection directory.

#### Scenario: Report refresh
- **WHEN** the report refresh service runs
- **THEN** it scans central collected sessions and writes the latest JSON report atomically

#### Scenario: Dashboard health
- **WHEN** the dashboard service is running
- **THEN** `/healthz` returns `ok`

### Requirement: LAN Access
The system SHALL provide a documented `.lan` access path for the dashboard on the home network.

#### Scenario: Caddy available
- **WHEN** Orange Pi Caddy can be updated
- **THEN** `codex-usage.lan` proxies to the LXSO dashboard service

#### Scenario: Caddy not writable
- **WHEN** Caddy cannot be updated non-interactively
- **THEN** a user-level high-port `.lan` proxy can expose the dashboard without root access

### Requirement: Observable Logs
The system SHALL emit structured logs for collector runs, report refreshes, and dashboard HTTP access to local files and optionally to syslog and Loki.

#### Scenario: Collector log forwarding
- **WHEN** a collector run finishes
- **THEN** a structured event includes host, matched files, staged files, pushed files, skipped files, and errors

#### Scenario: Dashboard access log forwarding
- **WHEN** the dashboard handles an HTTP request
- **THEN** a structured access event can be forwarded to the configured syslog and Loki endpoints

#### Scenario: Grafana log visibility
- **WHEN** LXSO2 Grafana has a Loki datasource
- **THEN** collector, report-refresh, and dashboard events can be queried by `job="codex-usage-profiler"` and `service`
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
## 1. OpenSpec

- [x] 1.1 Write proposal, design, and network collection spec
- [x] 1.2 Validate the OpenSpec change strictly

## 2. Collector

- [x] 2.1 Add a dependency-free collector CLI with default tool globs
- [x] 2.2 Stage files with metadata sidecars and incremental state
- [x] 2.3 Push to local or SSH destinations
- [x] 2.4 Emit local JSONL and optional syslog events

## 3. Central Services

- [x] 3.1 Add report refresh and dashboard service assets
- [x] 3.2 Add collector systemd and launchd service assets
- [x] 3.3 Add high-port LAN proxy asset for rootless `.lan` access

## 4. Tests

- [x] 4.1 Add collector unit tests
- [x] 4.2 Add dashboard syslog/access test coverage
- [x] 4.3 Run unit, E2E, and OpenSpec validation

## 5. Deployment

- [x] 5.1 Deploy central profiler to LXSO1
- [x] 5.2 Install collectors on reachable machines
- [x] 5.3 Expose a `.lan` URL
- [x] 5.4 Verify report refresh, dashboard health, and Grafana-visible logging

## 6. Paperclip Attribution Hardening

- [x] 6.1 Preserve collected `source_path` as first-class session metadata
- [x] 6.2 Add structured Paperclip identifiers to collector sidecars
- [x] 6.3 Add optional Paperclip topology snapshots for central reports
- [x] 6.4 Attribute Paperclip company, project, staff, and task from explicit metadata, source paths, snapshots, and workspace hints
- [x] 6.5 Add regression tests for collected sidecars, prompt preambles, and metadata snapshots
Loading
Loading