|
2 | 2 |
|
3 | 3 | Open-source framework for building site-specific MCP servers over real authenticated user browser sessions. Turns your logged-in web sessions into composable, testable AI tools — locally. |
4 | 4 |
|
5 | | -## Quick start |
| 5 | +## Repo layout |
6 | 6 |
|
7 | | -No runtime code yet. See SPEC.md for the full product spec. |
| 7 | +``` |
| 8 | +packages/ |
| 9 | + core/ @browserkit-dev/core — the framework (published to npm) |
| 10 | + adapter-hackernews/ gitignored — clone of browserkit-dev/adapter-hackernews |
| 11 | + adapter-linkedin/ gitignored — clone of browserkit-dev/adapter-linkedin |
| 12 | + adapter-rescue-flights/ personal adapter, local-only (never in browserkit-dev org) |
| 13 | +tests/ |
| 14 | + e2e/ Full-daemon smoke tests (require Chromium) |
| 15 | +.github/workflows/ |
| 16 | + ci.yml Unit tests (job: unit) + E2E smoke (job: e2e, needs: unit) |
| 17 | + release.yml Changesets publish pipeline — publishes @browserkit-dev/core on merge |
| 18 | + test-adapters.yml Matrix CI: tests all 5 external adapters after core releases |
| 19 | +``` |
8 | 20 |
|
9 | 21 | ## Build & test |
10 | 22 |
|
11 | | -TBD — stack not decided yet (TypeScript + Playwright or Python + Playwright). |
| 23 | +```bash |
| 24 | +pnpm install |
| 25 | +pnpm build # builds core + workspace adapters |
| 26 | +pnpm test # unit tests across all workspace packages (192 tests) |
| 27 | +pnpm test:e2e # E2E smoke tests (requires Chromium — run after pnpm build) |
| 28 | +pnpm lint # tsc --noEmit across all packages |
| 29 | +``` |
| 30 | + |
| 31 | +Individual packages: |
| 32 | +```bash |
| 33 | +pnpm --filter @browserkit-dev/core build |
| 34 | +pnpm --filter @browserkit-dev/core test |
| 35 | +``` |
12 | 36 |
|
13 | 37 | ## Architecture |
14 | 38 |
|
15 | | -- **Session Manager** — persistent auth browser processes |
16 | | -- **Site Adapter** — per-site structured action modules |
17 | | -- **MCP Server** — wraps adapters as typed MCP tools |
18 | | -- **Human Handoff** — foregrounds browser for 2FA/confirmation |
19 | | -- See `ARCH.md` for full architecture and file tree |
20 | | -- See `SPEC.md` for detailed product spec |
| 39 | +- **`SessionManager`** — owns all browser contexts (one per adapter site), handles persistent profiles, headed/headless switching, storage-state injection |
| 40 | +- **`SiteAdapter` interface** — adapters implement `site`, `domain`, `loginUrl`, `isLoggedIn()`, `tools()` — optional: `preparePage()`, `getLoginOptions()`, `minCoreVersion`, `requirements` |
| 41 | +- **`createAdapterServer`** — wraps an adapter as an HTTP MCP server; each connecting client gets its own `McpServer + StreamableHTTPServerTransport` |
| 42 | +- **`LockManager`** — FIFO serialization of tool calls per site; `holdForUser` for paused mode |
| 43 | +- **`withLoginFlow`** — opt-in automated form-fill login; falls back to human-handoff if not configured |
| 44 | +- See `ARCH.md` for full file tree, `SPEC.md` for product requirements |
| 45 | + |
| 46 | +## External adapter repos (all under `browserkit-dev/`) |
| 47 | + |
| 48 | +All 5 adapters are published on npm at `@browserkit-dev/adapter-*`: |
| 49 | + |
| 50 | +| Adapter | Repo | npm | |
| 51 | +|---------|------|-----| |
| 52 | +| HackerNews | `browserkit-dev/adapter-hackernews` | `@browserkit-dev/adapter-hackernews@0.1.0` | |
| 53 | +| Google Discover | `browserkit-dev/adapter-google-discover` | `@browserkit-dev/adapter-google-discover@0.1.0` | |
| 54 | +| LinkedIn | `browserkit-dev/adapter-linkedin` | `@browserkit-dev/adapter-linkedin@0.1.0` | |
| 55 | +| Reddit | `browserkit-dev/adapter-reddit` | `@browserkit-dev/adapter-reddit@0.1.0` | |
| 56 | +| Booking.com | `browserkit-dev/adapter-booking` | `@browserkit-dev/adapter-booking@0.1.0` | |
| 57 | + |
| 58 | +Local dev: clone any adapter into `packages/` — pnpm workspace links `@browserkit-dev/core` automatically. |
| 59 | + |
| 60 | +## CI overview |
| 61 | + |
| 62 | +``` |
| 63 | +browserkit-dev/browserkit ci.yml |
| 64 | + job: unit pnpm install → pnpm build → pnpm test |
| 65 | + job: e2e (needs: unit) install Chromium → build → checkout HN adapter → pnpm test:e2e |
| 66 | +
|
| 67 | +browserkit-dev/adapter-* ci.yml (all identical) |
| 68 | + npm ci → install Chromium → npm run build → npm test |
| 69 | +
|
| 70 | +browserkit-dev/browserkit release.yml |
| 71 | + On push to main: changesets/action → version bump PR or npm publish |
| 72 | +
|
| 73 | +browserkit-dev/browserkit test-adapters.yml |
| 74 | + After Release workflow succeeds: matrix job runs each adapter's CI against latest core |
| 75 | +``` |
| 76 | + |
| 77 | +**After every significant change, verify CI is green** before closing out a task: |
| 78 | +```bash |
| 79 | +for repo in browserkit adapter-hackernews adapter-google-discover adapter-linkedin adapter-reddit adapter-booking; do |
| 80 | + r=$(gh run list --repo browserkit-dev/$repo --workflow=ci.yml --limit 1 --json status,conclusion --jq '.[0] | "\(.status) \(.conclusion)"') |
| 81 | + echo "$repo: $r" |
| 82 | +done |
| 83 | +``` |
| 84 | + |
| 85 | +Known flaky: `e2e > get_top returns real HN articles` fails with `net::ERR_ABORTED` when GitHub IPs are rate-limited by HN — retrigger once if only this test fails with a network error. |
21 | 86 |
|
22 | 87 | ## Non-negotiables |
23 | 88 |
|
24 | | -- Strict typing — no `any` (TS) or untyped functions (Python) |
| 89 | +- Strict typing — no `any` (TS) |
25 | 90 | - Every new module must have tests |
26 | | -- Reference `SPEC.md` for product requirements |
27 | | -- Never cloud-sync credentials — local-only |
| 91 | +- No `console.log` in committed code (use `getLogger()` from `logger.ts`) |
| 92 | +- No hardcoded credentials — local-only, never committed |
| 93 | +- No new top-level folders without updating `ARCH.md` |
| 94 | +- Conventional commits: `feat:`, `fix:`, `refactor:`, `test:`, `docs:`, `chore:` |
| 95 | +- Every behavioral change needs a changeset (`pnpm changeset`) before merging to main |
28 | 96 |
|
29 | 97 | ## How to work here |
30 | 98 |
|
31 | | -- Small incremental commits with conventional messages |
32 | | -- Update `.context/progress.md` after completing tasks |
33 | | -- Update `.context/activeContext.md` when switching focus |
34 | | -- Run `/status` to see current project state |
35 | | -- Run `/plan` to decide what to work on next |
36 | | -- Run `/review` before committing |
| 99 | +- `AGENTS.md` — durable workspace facts; read before starting a session |
| 100 | +- `ARCH.md` — file tree and component map |
| 101 | +- `.context/progress.md` — update after completing tasks |
| 102 | +- `browserkit doctor --config browserkit.config.js` — check adapter compatibility |
37 | 103 |
|
38 | 104 | ## Forbidden patterns |
39 | 105 |
|
40 | | -- No `any` types (TS) or untyped functions (Python) |
| 106 | +- No `any` types |
41 | 107 | - No hardcoded credentials anywhere |
42 | | -- No `console.log` in committed code (use proper logging) |
43 | | -- No new top-level folders without updating ARCH.md |
| 108 | +- No `console.log` (use `log.info()` / `log.warn()` / `log.error()`) |
| 109 | +- No new top-level folders without updating `ARCH.md` |
| 110 | +- No pushing to main without verifying CI passes |
0 commit comments