You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
### `packages/adapter-rescue-flights` — new public adapter
6
-
7
-
Built a complete Phase 1+2 rescue flights adapter covering El Al and Israir:
8
-
9
-
-**El Al scraper** (`elal.ts`): Angular CDK virtual-scroll page. Scrolls in 300px steps and collects data at each position to handle DOM node recycling. Returns flight#, departure time, and exact seat count (0 = sold out) for all flights in the next 8 days.
- Phase 1: reads listing page DOM for all announced rescue flights (58+ cards, two directions).
12
-
- Phase 2: seeds a browser session, then calls Israir's internal `priceBar` API (`/api/results/priceBar`) per unique route for seat counts + prices across all dates. For available flights, intercepts the `FLIGHTS` API (`/api/search/FLIGHTS`) via `page.route()` to get exact flight#, departure time, and per-flight seat count.
13
-
- Total time: ~45s for from_tel_aviv (51 flights), ~15s for to_tel_aviv listing-only.
14
-
- All API calls via `page.evaluate()` — Imperva/TLS fingerprint bound, Node.js fetch rejected.
15
-
-**`run-check.ts`** and **`Makefile`**: `make agent-check` calls adapter tools over MCP, dumps `agent-check-results.json`. Agent does visual verification using browser screenshots.
16
-
-**14 unit tests** passing; integration tests updated for Phase 2 assertions.
17
-
-**`browserkit.config.js`** updated: port 52746.
18
-
-**`pnpm-workspace.yaml`** updated.
19
-
-**README.md** updated with rescue flights entry in Available Adapters table.
20
-
21
3
## Session 0 — Project Setup (2026-03-22)
22
4
23
5
- Synthesized SPEC.md from ChatGPT brainstorm conversation about authenticated MCP wrappers
Copy file name to clipboardExpand all lines: AGENTS.md
+2-11Lines changed: 2 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -30,7 +30,7 @@ Durable facts and correction patterns for this workspace. Updated by continual-l
30
30
- Browser mode switching (`headless` / `watch` / `paused`), screenshot, page state, and navigate are consolidated into a single `browser` MCP tool with an `action` parameter — user explicitly asked to reduce tool count ("too many tools"); do NOT revert to 5 separate management tools
31
31
- Management tools bypass the LockManager; regular automation tools go through it
32
32
- "Raw" Playwright access means exposing the CDP WebSocket URL (`wsEndpoint()`) of each adapter's browser — external agents (Claude Code, Cursor) attach to the already-authenticated session and write their own Playwright scripts via shell
33
-
- The Playwright skill pattern: AI writes a script to `/tmp`, executes it via shell — primary approach for shell-capable clients (Cursor, Claude Code); `run_script` MCP tool **implemented** using `quickjs-emscripten` (sandboxed QuickJS) for clients without shell access (Claude Desktop)
33
+
- The Playwright skill pattern: AI writes a script to `/tmp`, executes it via shell — primary approach for shell-capable clients (Cursor, Claude Code); opt-in `run_script` MCP tool planned for clients without shell access (Claude Desktop)
34
34
- MCP resources use `page://${site}/snapshot` (site name dynamic) — user pushed back when the URI appeared to hardcode the adapter name
35
35
- Testing utilities (`createTestAdapterServer`, `createTestMcpClient`) live at `@browserkit/core/testing` subpath — a separate harness package was explicitly rejected ("I don't think we need it, it should be in either adapter or in core")
36
36
- Real Chrome (`channel: "chrome"`) is required for Google-based adapters — Playwright's bundled Chromium is blocked by Google's login with "This browser or app may not be secure". `isLoggedIn` must NOT navigate during login polling or it redirects the user away from the sign-in page.
@@ -41,8 +41,7 @@ Durable facts and correction patterns for this workspace. Updated by continual-l
41
41
- Framework navigates to `adapter.loginUrl` before calling `isLoggedIn()` when browser is at `about:blank` — adapters do NOT need to handle this themselves
42
42
-`warm_up_browser()` (visiting google/wiki/github before login) was evaluated from stickerdaniel's code — decided as "nice to have" for first-time login, not adopted yet
43
43
-`browserkit login <site>` is blocked by the `CI=1` env var that Cursor sets — must run as `CI="" node packages/core/dist/cli.js login <site>` to open a headed browser from within Cursor terminal
44
-
-`browser` tool `snapshot` action is **implemented** — returns incremental aria-snapshot diff (`track: boolean` returns only lines changed since last call via set-difference); `page-snapshot` MCP resource shares the same `captureSnapshot()` helper. Inspired by `SawyerHood/dev-browser`.
45
-
-`browserkit login --connect` flag is **implemented** — attaches to an existing CDP endpoint via `autoDiscoverCdpEndpoint()` in `session-manager.ts`; useful when Chrome is already running.
44
+
-`browser` tool `snapshot` action is planned — returns incremental aria-snapshot diff, more token-efficient than screenshots; `page-snapshot` MCP resource already exists, the action adds diff support. Inspired by `SawyerHood/dev-browser`.
46
45
47
46
## Design Process Preferences
48
47
@@ -62,14 +61,6 @@ Durable facts and correction patterns for this workspace. Updated by continual-l
62
61
- Adapter roadmap priority: Reddit → Twitter/X (flagship, hardest bot detection) → Amazon (no consumer API at all) → Airbnb, Google Maps, Booking.com — all documented in the main README as planned adapters.
63
62
- Reddit adapter is two-phase: Phase 1 = unauthenticated `old.reddit.com` (stable HTML class names, no login required); Phase 2 = authenticated (separate plan file). Target `old.reddit.com` exclusively — new Reddit is a React SPA with aggressive DOM churn.
64
63
- Booking.com adapter was started (plan + Phase 1 architecture); user preference is to plan architecture first, then implement.
65
-
- Booking.com adapter has **6 tools**: `get_upcoming_bookings`, `get_past_bookings`, `get_booking_details` (account tools) + `search_hotels`, `get_property`, `get_availability` (search/property tools). All 6 work in **headless mode** via CloakBrowser — no watch mode required.
66
-
-**CloakBrowser** (`cloakbrowser` npm package) is in `@browserkit/core``optionalDependencies` — lazily imported only when `antiDetection: { useCloakBrowser: true }` is set in adapter config. Provides 33 C++-level stealth patches that bypass DataDome. Do NOT use `channel: "chrome"` for Booking.com; use `useCloakBrowser: true` instead.
67
-
- Global anti-detection patches applied to **all adapters** (unconditionally): `viewport: null` (avoids 1280×720 bot signal), `--blink-settings=primaryPointerType=4,...` (declares mouse at Blink level), `--window-size=1920,1080` (non-zero headless dimensions).
68
64
- Phased adapter development pattern: Phase 1 is unauthenticated/mock, Phase 2 is authenticated/live. Add verification gates between phases.
69
65
- Live scraping tests should run in GitHub CI via an external browser service (desired; specific service not yet chosen).
70
66
- The main browserkit README doubles as the project's public-facing "blogpost" — user refers to it interchangeably; keep it polished and up-to-date with available + planned adapters
71
-
-`adapter-rescue-flights`**has been implemented** (Phase 1 complete): scrapes El Al (`elal.com/heb/seat-availability?d=0`/`d=1`) via Angular DOM (`.seat-availability-list > .flight-group > .flight-item`) and Israir (`israir.co.il/Flights/Rescue_Flights/To_Tel_Aviv` / `/From_Tel_Aviv`) via internal APIs; both sites are public/unauthenticated.
72
-
- Israir listing page shows flights as `button.flight-promotion.manual` elements — buttons have **no URLs and no data attributes**; clicking them opens a SPA booking widget, NOT a new page. Do NOT try to extract URLs from them.
73
-
- Israir internal APIs discovered: `GET /api/results/priceBar?origin=TLV&destination=LCA&isOneWay=false&adults=1` (per-date seat counts + cheapest price) and `POST /api/search/FLIGHTS?origin=...&destination=...&startDate=...` with body `{"ignoredResults":0,"siteId":"isra2023"}` (flight number, departure/arrival times, exact seat count per segment).
74
-
- Israir APIs require Imperva/Incapsula session cookies (`visid_incap_*`, `incap_ses_*`, `reese84`) bound to the browser's TLS fingerprint — Node.js `fetch` with copied cookies is rejected. Must call via `page.evaluate()` from within the browser context.
75
-
- Israir scraping pattern (45s total vs 4 min naive): (1) read listing page DOM for all `button.flight-promotion.manual` cards, (2) seed-navigate to one search URL to establish Imperva session, (3) call `priceBar` API for all 6 destination cities in parallel via `page.evaluate()`, (4) intercept `POST /api/search/FLIGHTS` via route interception on 4 concurrent browser context pages for available dates only.
0 commit comments