Skip to content

Commit 04b52bd

Browse files
jonzareckijonzarecki
committed
feat(release): add changesets, simplify CI, publish @browserkit/core to npm
- Add @changesets/cli + .changeset/config.json for versioning - Add .github/workflows/release.yml for automated npm publishing - Move patchright from dependencies to peerDependencies in core (fixes TS type duplication) - Remove @browserkit/adapter-hackernews from core optionalDependencies - Remove postinstall script from core (patchright install now handled by consumers) - Simplify CI: remove monkey-patching E2E steps (workspace adapter handles E2E) - Fix local adapter peer deps workspace:* → ^0.1.0 in package.json - Add CONTRIBUTING.md with local dev and release instructions Made-with: Cursor
1 parent d609645 commit 04b52bd

7 files changed

Lines changed: 855 additions & 162 deletions

File tree

‎.changeset/config.json‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"$schema": "https://unpkg.com/@changesets/config@3.0.0/schema.json",
3+
"changelog": "@changesets/cli/changelog",
4+
"commit": false,
5+
"fixed": [],
6+
"linked": [],
7+
"access": "public",
8+
"baseBranch": "main",
9+
"updateInternalDependencies": "patch",
10+
"ignore": []
11+
}

‎.github/workflows/ci.yml‎

Lines changed: 0 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -34,29 +34,6 @@ jobs:
3434
- name: Test (unit)
3535
run: pnpm test
3636

37-
# E2E: checkout the HN adapter, patch its core dep, build it, then run E2E
38-
- name: Checkout adapter-hackernews for E2E
39-
uses: actions/checkout@v4
40-
with:
41-
repository: browserkit-dev/adapter-hackernews
42-
path: packages/adapter-hackernews
43-
44-
- name: Build adapter-hackernews for E2E
45-
working-directory: packages/adapter-hackernews
46-
run: |
47-
node -e "
48-
const pkg = JSON.parse(require('fs').readFileSync('package.json'));
49-
pkg.devDependencies['@browserkit/core'] = 'file:../core';
50-
pkg.peerDependencies['@browserkit/core'] = '>=0.1.0';
51-
// Pin patchright to the exact version installed in core so npm and
52-
// pnpm resolve the same Page type — prevents TS2322 on handler signatures.
53-
const corePr = JSON.parse(require('fs').readFileSync('../core/node_modules/patchright/package.json'));
54-
pkg.devDependencies['patchright'] = corePr.version;
55-
require('fs').writeFileSync('package.json', JSON.stringify(pkg, null, 2));
56-
"
57-
npm install
58-
npm run build
59-
6037
- name: Test (E2E smoke)
6138
run: pnpm test:e2e
6239
timeout-minutes: 5

‎.github/workflows/release.yml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
branches: [main]
6+
7+
concurrency: ${{ github.workflow }}-${{ github.ref }}
8+
9+
jobs:
10+
release:
11+
name: Release
12+
runs-on: ubuntu-latest
13+
14+
steps:
15+
- uses: actions/checkout@v4
16+
with:
17+
# Full history so changeset can compare against tags
18+
fetch-depth: 0
19+
20+
- uses: pnpm/action-setup@v4
21+
22+
- uses: actions/setup-node@v4
23+
with:
24+
node-version: 20
25+
cache: pnpm
26+
registry-url: https://registry.npmjs.org
27+
28+
- name: Install dependencies
29+
run: pnpm install --frozen-lockfile
30+
31+
- name: Build
32+
run: pnpm build
33+
34+
- name: Create Release PR or Publish
35+
id: changesets
36+
uses: changesets/action@v1
37+
with:
38+
publish: pnpm changeset publish
39+
version: pnpm changeset version
40+
title: "chore: release packages"
41+
commit: "chore: release packages"
42+
env:
43+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
44+
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

‎CONTRIBUTING.md‎

Lines changed: 51 additions & 129 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,15 @@
11
# Contributing to browserkit
22

3-
## Overview
3+
## Prerequisites
44

5-
browserkit is a framework for building site-specific MCP servers over real authenticated browser sessions. This guide covers how to contribute adapters, extend the core, and maintain code quality.
5+
- [Node.js](https://nodejs.org) >= 20
6+
- [pnpm](https://pnpm.io) >= 10 (`npm install -g pnpm`)
67

7-
## Development Setup
8+
---
9+
10+
## Working on core only
11+
12+
Clone the monorepo, install, build, and test:
813

914
```bash
1015
git clone https://github.com/browserkit-dev/browserkit
@@ -14,158 +19,75 @@ pnpm build
1419
pnpm test
1520
```
1621

17-
## Scraping Philosophy: Stable Over Clever
18-
19-
The most important principle for adapter development: **favour stability over specificity**.
22+
---
2023

21-
Google changes `.DY5T1d` class names constantly. LinkedIn redesigns component structure every few months. The adapters that survive these changes anchor on stable signals:
24+
## Working on core + an adapter simultaneously
2225

23-
**Prefer structural/semantic selectors over class names:**
26+
Adapter directories are gitignored in this repo — they live in their own GitHub repositories under [browserkit-dev](https://github.com/browserkit-dev). Clone the adapter you want into the `packages/` folder and pnpm will link `@browserkit/core` to your local workspace automatically — no `npm link`, no `file:`, no manual steps.
2427

25-
```typescript
26-
// ✅ Stable — Google uses data-hveid for click tracking internally
27-
"[data-hveid]"
28+
```bash
29+
# Clone the monorepo
30+
git clone https://github.com/browserkit-dev/browserkit
31+
cd browserkit
2832

29-
// ✅ Stable — heading roles are semantic, not layout
30-
"h3, [role='heading']"
33+
# Clone the adapter you want to work on into packages/
34+
git clone https://github.com/browserkit-dev/adapter-hackernews packages/adapter-hackernews
3135

32-
// ❌ Fragile — will break when Google pushes a CSS update
33-
".DY5T1d-RZkmj"
34-
```
36+
# A single pnpm install links everything
37+
pnpm install
3538

36-
**Prefer `textContent` extraction over DOM walking:**
39+
# Build core then the adapter
40+
pnpm build
3741

38-
```typescript
39-
// ✅ Resilient — extracts longest text node as title (works even if markup changes)
40-
const allText = Array.from(card.querySelectorAll("div, span"))
41-
.filter(el => el.children.length === 0)
42-
.map(el => el.textContent.trim())
43-
.filter(t => t.length > 20);
44-
const title = allText.sort((a, b) => b.length - a.length)[0];
42+
# Run all unit tests (core + adapter)
43+
pnpm test
4544

46-
// ❌ Fragile — breaks when class name changes
47-
card.querySelector(".article-title-class").textContent
45+
# Run E2E smoke tests
46+
pnpm test:e2e
4847
```
4948

50-
**Prefer URL navigation over clicking:**
51-
52-
```typescript
53-
// ✅ Navigate directly to the section URL
54-
await page.goto("https://linkedin.com/in/username/details/experience/");
49+
Any change you make to `packages/core/src/` is visible in the adapter immediately after `pnpm --filter @browserkit/core build`.
5550

56-
// ❌ Click a tab element whose selector may change
57-
await page.click('[data-section="experience"]');
58-
```
51+
### Available adapters
5952

60-
**Document any DOM dependency** with a comment explaining why `textContent`/URL navigation isn't sufficient.
53+
| Adapter | Repo |
54+
|---------|------|
55+
| HackerNews | [browserkit-dev/adapter-hackernews](https://github.com/browserkit-dev/adapter-hackernews) |
56+
| LinkedIn | [browserkit-dev/adapter-linkedin](https://github.com/browserkit-dev/adapter-linkedin) |
57+
| Google Discover | [browserkit-dev/adapter-google-discover](https://github.com/browserkit-dev/adapter-google-discover) |
58+
| Reddit | [browserkit-dev/adapter-reddit](https://github.com/browserkit-dev/adapter-reddit) |
59+
| Booking.com | [browserkit-dev/adapter-booking](https://github.com/browserkit-dev/adapter-booking) |
6160

6261
---
6362

64-
## Checklist: Building a New Adapter
65-
66-
### Scaffold
67-
68-
```bash
69-
npx @browserkit/core create-adapter my-site
70-
cd adapter-my-site
71-
pnpm install
72-
```
73-
74-
### Code
75-
76-
- [ ] `src/selectors.ts` — CSS selector constants with stability comments; prefer `data-*` attributes and semantic elements over class names
77-
- [ ] `src/index.ts`:
78-
- [ ] `defineAdapter({ site, domain, loginUrl, selectors, rateLimit })` — set `rateLimit` for authenticated sites
79-
- [ ] `isLoggedIn(page)` — check current page only (no navigation); return `true` for public sites
80-
- [ ] `tools()` — one tool per distinct URL/action; include `annotations: { readOnlyHint, openWorldHint }`
81-
- [ ] Tool handlers return `{ content, references? }` — include `references` for extractable links
82-
- [ ] Extract complex scraping logic into a `src/scraper.ts` function that takes a `Page` and can be tested independently
63+
## Releasing a new version of `@browserkit/core`
8364

84-
### Tests
65+
This repo uses [Changesets](https://github.com/changesets/changesets) for versioning and publishing.
8566

86-
- [ ] `tests/<site>.test.ts` — L1 unit: schema validation, metadata, selectors exported
87-
- [ ] `tests/<site>.scraping.test.ts` — mock DOM tests using a local HTML fixture; launch real Playwright browser against `file://` fixture URL; no network
88-
- [ ] `tests/mcp-protocol.test.ts` — L3: MCP initialize, tool list (includes `browser` + `close_session`), tool dispatch, `isError` on schema violations
89-
- [ ] `tests/reliability.test.ts` — L4: concurrency, latency p50/p95, error recovery
90-
- [ ] `tests/<site>.integration.test.ts` — L2: live scraping; tagged so default `pnpm test` excludes it
91-
92-
### Docs
93-
94-
- [ ] `README.md` — tool table, installation, config example with `deviceEmulation`/`channel` if needed
95-
- [ ] Update `browserkit.config.js` example in the main repo README if relevant
96-
97-
### Verify
67+
**On every PR that changes core:**
9868

9969
```bash
100-
pnpm build
101-
pnpm test # L1 + mock scraping + L3 + L4
102-
pnpm test:integration # L2 live (requires auth if applicable)
70+
pnpm changeset
71+
# Follow the prompt: choose "patch" / "minor" / "major", write a summary
72+
git add .changeset/
73+
git commit -m "chore: add changeset"
10374
```
10475

105-
---
106-
107-
## Checklist: Adding a Tool to an Existing Adapter
76+
When the PR is merged to `main`, the Release GitHub Action automatically:
77+
1. Opens (or updates) a "Release PR" that bumps the version and updates `CHANGELOG.md`
78+
2. When that Release PR is merged, publishes `@browserkit/core` to npm
10879

109-
- [ ] Add tool to `tools()` array in `src/index.ts`
110-
- [ ] Include `annotations: { readOnlyHint, openWorldHint }` (readOnly=false + destructiveHint=true for write operations)
111-
- [ ] Add `references` to the return value if the result contains navigable links
112-
- [ ] Update `src/selectors.ts` if new selectors are needed
113-
- [ ] Add L1 schema test covering the new input
114-
- [ ] Add mock scraping test if the tool involves DOM extraction
115-
- [ ] Update `README.md` tool table
80+
An `NPM_TOKEN` secret must be set in the repository settings for publishing to work.
11681

11782
---
11883

119-
## Tool Annotations Reference
84+
## Adapter development
12085

121-
All adapter tools should declare annotations so AI agents can reason about safety:
122-
123-
```typescript
124-
{
125-
name: "get_posts",
126-
annotations: {
127-
readOnlyHint: true, // tool does not modify state
128-
openWorldHint: true, // tool calls external services (always true for web adapters)
129-
},
130-
// ...
131-
}
132-
```
133-
134-
| Annotation | When to use |
135-
|---|---|
136-
| `readOnlyHint: true` | Any read-only scraping tool |
137-
| `readOnlyHint: false` + `destructiveHint: true` | Posting, deleting, purchasing |
138-
| `openWorldHint: true` | Any tool that hits a real website (always) |
139-
140-
---
86+
See any existing adapter as a reference — HackerNews is the simplest:
87+
[browserkit-dev/adapter-hackernews](https://github.com/browserkit-dev/adapter-hackernews)
14188

142-
## References in Tool Results
89+
Scaffold a new adapter with:
14390

144-
If a tool result contains links to other entities (articles, profiles, comments), include them in `references`:
145-
146-
```typescript
147-
return {
148-
content: [{ type: "text", text: JSON.stringify(articles) }],
149-
references: articles.map(a => ({
150-
kind: "article",
151-
url: a.url,
152-
text: a.title,
153-
context: a.source,
154-
})),
155-
};
91+
```bash
92+
npx @browserkit/core create-adapter my-site
15693
```
157-
158-
This allows AI agents to follow links without parsing JSON from the main content string.
159-
160-
---
161-
162-
## Code Style
163-
164-
- **No `any` types** — strict TypeScript throughout
165-
- **No `console.log`** in committed code — use `getLogger()` from core
166-
- **Conventional commits**: `feat:`, `fix:`, `refactor:`, `test:`, `docs:`, `chore:`
167-
- **Small incremental changes** — PRs should be reviewable in under 10 minutes
168-
169-
## Architecture
170-
171-
See [ARCH.md](ARCH.md) for the full framework architecture and open design questions.

‎package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
},
1313
"devDependencies": {
1414
"@browserkit/core": "workspace:*",
15+
"@changesets/cli": "^2.30.0",
1516
"typescript": "^5.7.3",
1617
"vitest": "^3.0.9"
1718
},

‎packages/core/package.json‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,15 @@
55
"type": "module",
66
"main": "./dist/index.js",
77
"types": "./dist/index.d.ts",
8+
"repository": {
9+
"type": "git",
10+
"url": "https://github.com/browserkit-dev/browserkit.git",
11+
"directory": "packages/core"
12+
},
13+
"publishConfig": {
14+
"access": "public",
15+
"registry": "https://registry.npmjs.org"
16+
},
817
"exports": {
918
".": {
1019
"import": "./dist/index.js",
@@ -23,19 +32,21 @@
2332
"build:watch": "tsc --watch",
2433
"test": "vitest run",
2534
"test:watch": "vitest",
26-
"lint": "tsc --noEmit",
27-
"postinstall": "patchright install chromium --with-deps || true"
35+
"lint": "tsc --noEmit"
36+
},
37+
"peerDependencies": {
38+
"patchright": "^1.51.0"
2839
},
2940
"dependencies": {
3041
"@modelcontextprotocol/sdk": "^1.10.2",
31-
"patchright": "^1.51.1",
3242
"pino": "^9.6.0",
3343
"playwright-core": "^1.58.2",
3444
"quickjs-emscripten": "^0.32.0",
3545
"zod": "^3.24.2"
3646
},
3747
"devDependencies": {
3848
"@types/node": "^22.13.14",
49+
"patchright": "^1.51.1",
3950
"tsx": "^4.19.3",
4051
"typescript": "^5.7.3",
4152
"vitest": "^3.0.9"
@@ -48,7 +59,6 @@
4859
"README.md"
4960
],
5061
"optionalDependencies": {
51-
"@browserkit/adapter-hackernews": "workspace:*",
5262
"cloakbrowser": "^0.3.18"
5363
}
5464
}

0 commit comments

Comments
 (0)