Skip to content

Commit 700445e

Browse files
pyramationpyramation
committed
docs: add AGENTS.md and fixtures testing skill
Add AGENTS.md with repo overview, package listing, setup instructions, and code conventions. Add .agents/skills/testing-fixtures/SKILL.md documenting the full fixture-based testing pipeline: how SQL fixtures get split into generated.json, how kitchen-sink tests are generated, and how to add new fixtures.
1 parent 8156070 commit 700445e

2 files changed

Lines changed: 222 additions & 0 deletions

File tree

‎.agents/skills/testing-fixtures/SKILL.md‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# pgsql-parser Fixtures & Testing System
2+
3+
This skill documents how the fixture-based testing pipeline works in pgsql-parser and how to add new test fixtures.
4+
5+
## Architecture Overview
6+
7+
The testing pipeline has three layers:
8+
9+
1. **SQL fixture files** — Raw SQL statements in `__fixtures__/kitchen-sink/`
10+
2. **Generated fixture JSON** — `__fixtures__/generated/generated.json` (individual statements keyed by path)
11+
3. **Generated test files** — `packages/deparser/__tests__/kitchen-sink/*.test.ts`
12+
13+
## Directory Structure
14+
15+
```
16+
__fixtures__/
17+
kitchen-sink/ # Source SQL fixture files
18+
latest/postgres/ # PostgreSQL regression test extracts
19+
misc/ # Miscellaneous test cases (issues, features)
20+
original/ # Original upstream PostgreSQL test files
21+
pretty/ # Pretty-print specific test cases
22+
generated/
23+
generated.json # Auto-generated: individual statements keyed by relative path
24+
plpgsql/ # PL/pgSQL fixture SQL files
25+
plpgsql-generated/
26+
generated.json # Auto-generated: PL/pgSQL fixtures
27+
packages/deparser/
28+
__tests__/
29+
kitchen-sink/ # Auto-generated test files from kitchen-sink fixtures
30+
misc/ # Hand-written test files for specific features
31+
pretty/ # Pretty-print tests using PrettyTest utility
32+
scripts/
33+
make-fixtures.ts # Splits SQL files -> generated.json
34+
make-kitchen-sink.ts # Generates kitchen-sink test files
35+
statement-splitter.ts # Extracts individual statements from multi-statement SQL
36+
test-utils/
37+
index.ts # Core test utilities (expectParseDeparse, FixtureTestUtils, TestUtils)
38+
PrettyTest.ts # Pretty-print test helper
39+
```
40+
41+
## How Fixtures Work
42+
43+
### Step 1: Write SQL fixture file
44+
45+
Create a `.sql` file in `__fixtures__/kitchen-sink/`. Organize by category:
46+
- `misc/` for bug fixes, feature-specific tests, issue reproductions
47+
- `pretty/` for pretty-print formatting tests
48+
- `latest/postgres/` for PostgreSQL regression test extracts
49+
- `original/` for original upstream PostgreSQL test files
50+
51+
Each SQL statement in the file becomes a separate test case. Use comments for context:
52+
53+
```sql
54+
-- Brief description of what's being tested
55+
-- Ref: constructive-io/pgsql-parser#289
56+
SELECT EXTRACT(EPOCH FROM now());
57+
SELECT EXTRACT(YEAR FROM TIMESTAMP '2001-02-16 20:38:40');
58+
```
59+
60+
### Step 2: Regenerate fixtures
61+
62+
From `packages/deparser/`:
63+
64+
```bash
65+
# Generate generated.json from all kitchen-sink SQL files
66+
npm run fixtures
67+
68+
# Generate kitchen-sink test files from generated.json
69+
npm run fixtures:kitchen-sink
70+
71+
# Or do both at once:
72+
npm run kitchen-sink
73+
```
74+
75+
**What `make-fixtures.ts` does:**
76+
- Reads all `__fixtures__/kitchen-sink/**/*.sql` files
77+
- Splits each file into individual statements using `statement-splitter.ts`
78+
- Validates each statement parses correctly via `libpg-query`
79+
- Writes all statements to `__fixtures__/generated/generated.json` as `{"relative/path-N.sql": "SQL statement"}`
80+
81+
**What `make-kitchen-sink.ts` does:**
82+
- Reads all `__fixtures__/kitchen-sink/**/*.sql` files
83+
- For each file, generates a test file in `packages/deparser/__tests__/kitchen-sink/`
84+
- Test file name: `{category}-{name}.test.ts` (slashes become hyphens)
85+
- Each test file uses `FixtureTestUtils.runFixtureTests()` with the list of statement keys
86+
87+
### Step 3: Run tests
88+
89+
```bash
90+
# Run all deparser tests
91+
cd packages/deparser && npx jest
92+
93+
# Run only kitchen-sink tests
94+
npx jest __tests__/kitchen-sink/
95+
96+
# Run a specific fixture test
97+
npx jest __tests__/kitchen-sink/misc-extract.test.ts
98+
```
99+
100+
## Test Verification
101+
102+
The `FixtureTestUtils.runFixtureTests()` method (via `TestUtils.expectAstMatch()`) performs this verification for each statement:
103+
104+
1. **Parse** the original SQL -> AST
105+
2. **Deparse** (non-pretty) the AST -> SQL string
106+
3. **Reparse** the deparsed SQL -> new AST
107+
4. **Compare** the cleaned ASTs — they must match
108+
5. **Repeat** steps 2-4 with `pretty: true`
109+
110+
This ensures full round-trip fidelity: `parse(sql) -> deparse(ast) -> parse(deparsed) === original AST`
111+
112+
The `expectParseDeparse()` utility in `test-utils/index.ts` does the same thing for hand-written tests.
113+
114+
## Pretty-Print Tests
115+
116+
Pretty-print tests use a different pattern via `PrettyTest` class in `test-utils/PrettyTest.ts`:
117+
118+
```typescript
119+
import { PrettyTest } from '../../test-utils/PrettyTest';
120+
121+
const testCases = [
122+
'pretty/misc-1.sql',
123+
'pretty/misc-2.sql',
124+
];
125+
126+
const prettyTest = new PrettyTest(testCases);
127+
prettyTest.generateTests();
128+
```
129+
130+
This generates two tests per case (pretty and non-pretty) and uses Jest snapshots. Pretty tests read from `generated.json` and compare output via `toMatchSnapshot()`.
131+
132+
## Adding a New Fixture (Quick Reference)
133+
134+
1. Create/edit a `.sql` file in `__fixtures__/kitchen-sink/{category}/`
135+
2. Run `cd packages/deparser && npm run kitchen-sink`
136+
3. Run `npx jest` to verify all tests pass
137+
4. Commit the `.sql` file, `generated.json`, and any new generated test files in `__tests__/kitchen-sink/`
138+
139+
## Package Scripts Reference
140+
141+
From `packages/deparser/package.json`:
142+
143+
| Script | Command | Description |
144+
|--------|---------|-------------|
145+
| `fixtures` | `ts-node scripts/make-fixtures.ts` | Regenerate `generated.json` |
146+
| `fixtures:kitchen-sink` | `ts-node scripts/make-kitchen-sink.ts` | Regenerate kitchen-sink test files |
147+
| `kitchen-sink` | `npm run fixtures && npm run fixtures:kitchen-sink` | Both steps combined |
148+
| `fixtures:ast` | `ts-node scripts/make-fixtures-ast.ts` | Generate AST JSON fixtures (from legacy) |
149+
| `fixtures:sql` | `ts-node scripts/make-fixtures-sql.ts` | Generate SQL fixtures via native deparse (from legacy) |
150+
| `fixtures:upstream-diff` | `ts-node scripts/make-upstream-diff.ts` | Generate diff of upstream vs deparsed |
151+
| `test` | `jest` | Run all tests |
152+
| `test:watch` | `jest --watch` | Run tests in watch mode |
153+
154+
## Build & Lint
155+
156+
```bash
157+
# From repo root
158+
pnpm install
159+
pnpm run build
160+
161+
# From packages/deparser
162+
npm run build # tsc (CJS + ESM) + asset copy
163+
npm run lint # eslint --fix
164+
```

‎AGENTS.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# AGENTS.md — pgsql-parser
2+
3+
## Project Overview
4+
5+
A pnpm monorepo for PostgreSQL AST parsing, deparsing, and code generation. All packages live in `packages/`.
6+
7+
## Key Packages
8+
9+
| Package | Directory | Purpose |
10+
|---------|-----------|---------|
11+
| `pgsql-parser` | `packages/parser` | Parse SQL to AST (wraps `libpg-query` WASM) |
12+
| `pgsql-deparser` | `packages/deparser` | Convert AST back to SQL (pure TypeScript) |
13+
| `plpgsql-parser` | `packages/plpgsql-parser` | Parse PL/pgSQL to AST |
14+
| `plpgsql-deparser` | `packages/plpgsql-deparser` | Convert PL/pgSQL AST back to SQL |
15+
| `@pgsql/types` | `packages/pgsql-types` | TypeScript type definitions for PostgreSQL AST nodes |
16+
| `@pgsql/utils` | `packages/utils` | Type-safe AST node creation utilities |
17+
| `@pgsql/traverse` | `packages/traverse` | Visitor-pattern AST traversal |
18+
| `@pgsql/transform` | `packages/transform` | Multi-version AST transformer (PG 13→17) |
19+
| `@pgsql/quotes` | `packages/quotes` | SQL identifier/string quoting utilities |
20+
| `@pgsql/cli` | `packages/pgsql-cli` | CLI tool for parse/deparse operations |
21+
| `pg-proto-parser` | `packages/proto-parser` | Generate TypeScript from PostgreSQL protobuf definitions |
22+
| `pg-ast` | `packages/pg-ast` | Low-level AST types |
23+
24+
## Setup
25+
26+
```bash
27+
pnpm install
28+
pnpm run build # builds all packages
29+
pnpm run test # runs all package tests
30+
pnpm run lint # lints all packages
31+
```
32+
33+
## Per-Package Commands
34+
35+
Each package supports:
36+
- `npm run build` — TypeScript compilation (CJS + ESM) + asset copy
37+
- `npm run test` — Jest tests
38+
- `npm run lint` — ESLint with auto-fix
39+
- `npm run test:watch` — Jest in watch mode
40+
41+
## Testing
42+
43+
Tests use Jest. The deparser packages use a **fixture-based testing system** — see `.agents/skills/testing-fixtures/SKILL.md` for full details.
44+
45+
Quick reference:
46+
```bash
47+
cd packages/deparser
48+
npm run kitchen-sink # regenerate fixtures + test files
49+
npx jest # run all tests
50+
```
51+
52+
## Code Conventions
53+
54+
- TypeScript throughout, compiled to both CJS and ESM
55+
- `@pgsql/types` provides all AST node types — use them for type safety
56+
- `@pgsql/quotes` handles SQL identifier quoting — use `QuoteUtils` methods
57+
- Test files go in `__tests__/` within each package
58+
- Fixture SQL files go in `__fixtures__/kitchen-sink/` (see skills for details)

0 commit comments

Comments
 (0)