Skip to content

Commit 6939b4a

Browse files
nikosdouvlisnikosdouvlis
committed
fix: use correct pkglab registry port (16180) in health check
The global setup was checking port 4873 (old Verdaccio default) but pkglab runs on port 16180. Also adds integration/CLAUDE.md.
1 parent 26cca4f commit 6939b4a

2 files changed

Lines changed: 160 additions & 1 deletion

File tree

‎integration/CLAUDE.md‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
# Integration Test Infrastructure
2+
3+
This directory contains the E2E test suite for all Clerk JS SDKs. Tests run against real Clerk instances using real package tarballs installed via a local Verdaccio registry managed by pkglab.
4+
5+
## Key facts
6+
7+
- `integration/` is NOT a pnpm workspace member (not listed in `pnpm-workspace.yaml`). It gets `@clerk/backend` and other deps via root workspace hoisting.
8+
- Test apps are created in `os.tmpdir()/.temp_integration/` - completely outside the monorepo workspace.
9+
- `@clerk/backend` is used here purely as infrastructure (BAPI calls via `createClerkClient`) - it is not a test subject.
10+
- The test apps in `/tmp` are what actually verify SDK behavior with real tarballs.
11+
- Templates in `templates/` are copy sources only - they are never executed in-place.
12+
13+
## Directory layout
14+
15+
- `models/` - Core abstractions: `applicationConfig.ts`, `application.ts`, `longRunningApplication.ts`, `environment.ts`
16+
- `presets/` - Pre-built configs for each framework: `react.ts`, `next.ts`, `vue.ts`, etc. Also `envs.ts` (Clerk instance configs) and `longRunningApps.ts`
17+
- `templates/` - Static app source trees that get copied to `/tmp` during tests
18+
- `testUtils/` - `createTestUtils`, page objects, and services (`usersService`, `emailService`, `organizationsService`, etc.)
19+
- `tests/` - The actual Playwright test files
20+
21+
## How tests work
22+
23+
1. `pkglab pub` - builds and publishes all `@clerk/*` packages to a local Verdaccio registry
24+
2. `appConfigs.<framework>.<variant>.clone().commit()` - copies the template to `/tmp/.temp_integration/<name>__<timestamp>__<hash>/` and patches `package.json`
25+
3. `app.setup()` - runs `pkglab add @clerk/react @clerk/shared ...` which pins those deps to the local registry versions and installs everything
26+
4. `app.withEnv(appConfigs.envs.withEmailCodes)` - writes a `.env` file into the tmp dir with the Clerk instance keys
27+
5. `app.dev()` - starts the dev server, waits for it to be ready, sets up Clerk testing tokens
28+
6. Playwright tests run against `app.serverUrl`
29+
7. `app.teardown()` - kills the server and removes the tmp directory
30+
31+
## `linkPackage()` and `clerkDependencies`
32+
33+
`linkPackage()` in `presets/utils.ts` returns `'*'`. When `commit()` patches `package.json`, it writes `"@clerk/react": "*"`. Then `pkglab add` resolves `*` against the local Verdaccio registry, which has the locally-built tarballs. The `config.clerkDependencies` getter in `applicationConfig.ts` returns all dependencies whose name starts with `@clerk/`, which is the list passed to `pkglab add`.
34+
35+
## Long-running applications
36+
37+
Most tests use long-running apps defined in `presets/longRunningApps.ts`. These are started once in `global.setup.ts`, shared across all tests via a state file (`/tmp/.temp_integration/state.json`), and stopped in `global.teardown.ts`. This avoids re-installing deps for every test file.
38+
39+
Use `testAgainstRunningApps({ withEnv: [appConfigs.envs.withEmailCodes] })` in test files to run against these pre-started apps. Each long-running app has an id like `next.appRouter.withEmailCodes` - use `E2E_APP_ID` to filter which ones start.
40+
41+
## Environment configs (`envs`)
42+
43+
An environment config holds PK/SK pairs for a specific Clerk instance plus other env vars. Instance keys come from either:
44+
45+
- `.keys.json` in the `integration/` directory (local dev, gitignored)
46+
- `INTEGRATION_INSTANCE_KEYS` env var (CI, contains the same JSON)
47+
48+
The `base` config in `presets/envs.ts` sets shared vars like `CLERK_SIGN_IN_URL`, `CLERK_TELEMETRY_DISABLED`, and the clerk-js/clerk-ui URLs (defaulting to `http://localhost:18211/clerk.browser.js` for local dev).
49+
50+
## Environment variables
51+
52+
Important variables from `constants.ts`:
53+
54+
- `E2E_APP_ID` - which long-running apps to start, e.g. `next.appRouter.*` or `react.vite.withEmailCodes`
55+
- `E2E_APP_URL` / `E2E_APP_SK` / `E2E_APP_PK` - point tests at a manually running app
56+
- `E2E_APP_CLERK_JS` - override clerk-js URL (skip local clerk-js server)
57+
- `E2E_CLEANUP=0` - keep the `/tmp` app around after the test run for debugging
58+
- `E2E_NEXTJS_VERSION`, `E2E_REACT_VERSION`, etc. - pin specific upstream dep versions
59+
- `INTEGRATION_INSTANCE_KEYS` - JSON blob with all Clerk instance PK/SK pairs (used in CI)
60+
61+
## Running tests
62+
63+
All commands run from the repo root:
64+
65+
```sh
66+
# Prerequisite: publish packages to local registry
67+
pkglab pub
68+
69+
# Run all integration tests
70+
pnpm test:integration:base
71+
72+
# Run tests for a specific framework
73+
pnpm test:integration:nextjs
74+
pnpm test:integration:vue
75+
76+
# Filter by test file
77+
pnpm test:integration:base -- email-code.test.ts
78+
79+
# Debug mode
80+
pnpm test:integration:base -- --ui email-code.test.ts
81+
82+
# Keep tmp app after run (for inspection)
83+
E2E_CLEANUP=0 pnpm test:integration:base
84+
```
85+
86+
## Writing tests
87+
88+
A typical test file structure:
89+
90+
```ts
91+
import { test } from '@playwright/test';
92+
import type { Application } from '../models/application';
93+
import { appConfigs } from '../presets';
94+
import { createTestUtils } from '../testUtils';
95+
96+
test.describe('Feature name @nextjs', () => {
97+
test.describe.configure({ mode: 'parallel' });
98+
let app: Application;
99+
100+
test.beforeAll(async () => {
101+
app = await appConfigs.next.appRouter.clone().commit();
102+
await app.setup();
103+
await app.withEnv(appConfigs.envs.withEmailCodes);
104+
await app.dev();
105+
});
106+
107+
test.afterAll(async () => {
108+
await app.teardown();
109+
});
110+
111+
test('can sign in', async ({ page, context }) => {
112+
const u = createTestUtils({ app, page, context });
113+
await u.po.signIn.goTo();
114+
await u.po.signIn.waitForMounted();
115+
await u.po.signIn.signInWithEmailAndInstantPassword({ email: fakeUser.email, password: fakeUser.password });
116+
await u.po.expect.toBeSignedIn();
117+
});
118+
});
119+
```
120+
121+
Use `testAgainstRunningApps` instead when targeting long-running apps:
122+
123+
```ts
124+
testAgainstRunningApps({ withEnv: [appConfigs.envs.withEmailCodes] })('Feature @generic', ({ app }) => {
125+
test('...', async ({ page, context }) => {
126+
const u = createTestUtils({ app, page, context });
127+
// ...
128+
});
129+
});
130+
```
131+
132+
## Adding a new framework preset
133+
134+
1. Create a template directory under `templates/` with a working app that uses Clerk
135+
2. Register it in `templates/index.ts`
136+
3. Create a preset file under `presets/` (e.g. `presets/myframework.ts`) using `applicationConfig()`, `useTemplate()`, `setEnvFormatter()`, and `addDependency('@clerk/myframework', linkPackage('myframework'))`
137+
4. Export it from `presets/index.ts` via `appConfigs`
138+
5. Add long-running app entries to `presets/longRunningApps.ts` if needed
139+
6. Add a script to the root `package.json` like `test:integration:myframework`
140+
141+
## Adding a new Clerk instance (env config)
142+
143+
1. Create a new instance in the **Integration testing** Clerk org
144+
2. Add its keys to 1Password ("JS SDKs integration tests" note)
145+
3. Run `pnpm integration:secrets` to sync keys to `.keys.json`
146+
4. Add an entry to `.keys.json.sample` (placeholder, committed)
147+
5. Add the keys to the `INTEGRATION_INSTANCE_KEYS` secret in GitHub
148+
6. Create an `environmentConfig()` entry in `presets/envs.ts` and export it from `envs`
149+
150+
## Debugging
151+
152+
If a test fails in CI but passes locally:
153+
154+
- Playwright traces are uploaded on failure - download and open at https://trace.playwright.dev/
155+
- Set `appDirName` to a stable name in `applicationConfig.ts` and upload the tmp dir as a CI artifact
156+
- Add `verbose: true` to the Turborepo setup step in `ci.yml` for full logs
157+
- Use `E2E_CLEANUP=0` locally to inspect the generated app after a failed run
158+
159+
The state file at `/tmp/.temp_integration/state.json` tracks running long-running apps between setup and teardown. If a test run is interrupted, stale entries may cause issues - delete the file or the whole `/tmp/.temp_integration/` directory to reset.

‎integration/tests/global.setup.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ setup('start long running apps', async () => {
1111

1212
// Verify pkglab registry is running
1313
try {
14-
execSync('curl -sf http://localhost:4873/ > /dev/null', { timeout: 5000 });
14+
execSync('curl -sf http://localhost:16180/ > /dev/null', { timeout: 5000 });
1515
} catch {
1616
throw new Error(
1717
'pkglab registry is not running. Start it with: pkglab pub\n' +

0 commit comments

Comments
 (0)