Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 88 additions & 55 deletions .github/workflows/build_and_test.yml

Large diffs are not rendered by default.

95 changes: 21 additions & 74 deletions docs/extensions/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,117 +108,63 @@ These steps allow our build environment to pick up the extension's code, then bu

#### Frontend

PGlite's frontend code is in the main [PGLite repo](https://github.com/electric-sql/pglite)
PGlite's frontend code is in the main [PGlite repo](https://github.com/electric-sql/pglite).

Create a new folder `packages/pglite/src/myawesomeextension` and a new file inside it `index.ts`. This is how PGlite will know how to load your new extension:

```
import type {
Extension,
ExtensionSetupResult,
PGliteInterface,
} from '../interface'

const setup = async (_pg: PGliteInterface, emscriptenOpts: any) => {
return {
emscriptenOpts,
bundlePath: new URL('../../release/myawesomeextension.tar.gz', import.meta.url),
} satisfies ExtensionSetupResult
}

export const myawesomeextension = {
name: 'myawesomeextension',
setup,
} satisfies Extension

```

Now add the extension to `packages/pglite/package.json` exports:

```
"exports": {
...
"./myawesomeextension": {
"import": {
"types": "./dist/myawesomeextension/index.d.ts",
"default": "./dist/myawesomeextension/index.js"
},
"require": {
"types": "./dist/myawesomeextension/index.d.cts",
"default": "./dist/myawesomeextension/index.cjs"
}
},
}
```

Open `packages/pglite/tsup.config.ts` and add your extension inside `entryPoints`:

```
const entryPoints = [
...
'src/myawesomeextension/index.ts'
]
```

You also need to add the extension to `packages/pglite/scripts/bundle-wasm.ts`, inside `main()`:

```
async function main() {
...
await findAndReplaceInDir('./dist/myawesomeextension', /\.\.\/release\//g, '', [
'.js',
'.cjs',
])
}
You will need to create a new package for your extension. The easiest way is to just copy the files from an existing extension package such as `packages/pglite-pgvector`. Obviously, you will need to adapt all the references to the previous extension.

```

To make it available in our online [REPL](https://pglite.dev/repl/), add the extension to `docs/repl/allExtensions.ts`:

```
export { myawesomeextension } from '@electric-sql/pglite/myawesomeextension

export { myawesomeextension } from '@electric-sql/pglite-myawesomeextension

```

Finally, add the extension description to `docs/extensions/extensions.data.ts`, inside `baseExtensions`:

```

const baseExtensions: Extension[] = [
...
{
name: 'My awesome Postgres extension',
description: `
My awesome Postgres extension is something that the world has never seen before.
name: 'My awesome Postgres extension',
description: ` My awesome Postgres extension is something that the world has never seen before.
`,
shortDescription:
'My awesome PostgreSQL extension',
docs: 'https://github.com/myawesomeextension/extension',
tags: ['postgres extension'],
importPath: '@electric-sql/pglite/myawesomeextension',
importName: 'my_awesome_extension',
size: 123456,
shortDescription:
'My awesome PostgreSQL extension',
docs: 'https://github.com/myawesomeextension/extension',
tags: ['postgres extension'],
importPath: '@electric-sql/pglite-myawesomeextension',
importName: 'my_awesome_extension',
size: 123456,
},
]

```

#### Tests

To make sure that your extension works, you need to add some tests for it. We use [vitest](https://vitest.dev/). They will be run as part of our CI/CD pipeline.
Add a file `packages/pglite/tests/myawesomeextension.test.ts` and write there your tests. Have a look inside that folder `packages/pglite/tests` at other tests to get an idea how they work.

#### Build and run tests

From PGlite's base folder:

```

$ pnpm build:all

```

This will build **everything**, including your new extension. If there are no errors, you are ready to run the tests!

```
$ cd packages/pglite

$ cd packages/pglite-myawesomextension
$ pnpm test

```

Fix any errors that occur, re-run the tests! Iterate until everything works as expected.
Expand All @@ -240,3 +186,4 @@ As mentioned before, some extensions require more effort to integrate with PGlit
Another source of pain for building an extension is the need to export symbols from the dependencies or from PGlite itself. Sometimes these are obvious only at runtime.

We are still working on documentation and examples showing how to build more complex Postgres extensions for use with PGlite. Please check back soon, or reach out on [Discord](https://discord.com/channels/933657521581858818/1212676471588520006) if you would like to try building a particular extension for PGlite.
```
16 changes: 16 additions & 0 deletions docs/extensions/extensions.data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -665,6 +665,22 @@ const baseExtensions: Extension[] = [
core: true,
size: 11554,
},
{
name: 'pgmq',
description: `
A lightweight message queue. Like AWS SQS and RSMQ but on Postgres.

This is an **external** extension, distributed as a separate package at https://www.npmjs.com/package/@electric-sql/pglite-pgmq
`,
shortDescription:
'A lightweight message queue. Like AWS SQS and RSMQ but on Postgres.',
docs: 'https://github.com/pgmq/pgmq',
tags: ['postgres extension', 'external'],
importPath: '@electric-sql/pglite-pgmq',
importName: 'pgmq',
size: 41908,
npmjsUrl: 'https://www.npmjs.com/package/@electric-sql/pglite-pgmq',
},
]

const tags = [
Expand Down
5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,16 @@
"ci:publish": "pnpm changeset publish",
"ts:build": "pnpm -r --filter \"./packages/**\" build",
"ts:build:debug": "DEBUG=true pnpm ts:build",
"wasm:copy-pgvector": "mkdir -p ./packages/pglite-pgvector/release && cp ./postgres-pglite/dist/extensions/other/vector.tar.gz ./packages/pglite-pgvector/release",
"wasm:copy-pgmq": "mkdir -p ./packages/pglite-pgmq/release && cp ./postgres-pglite/dist/extensions/other/pgmq-extension.tar.gz ./packages/pglite-pgmq/release/pgmq.tar.gz",
"wasm:copy-pgvector": "mkdir -p ./packages/pglite-pgvector/release && cp ./postgres-pglite/dist/extensions/other/vector.tar.gz ./packages/pglite-pgvector/release",
"wasm:copy-pg_uuidv7": "mkdir -p ./packages/pglite-pg_uuidv7/release && cp ./postgres-pglite/dist/extensions/other/pg_uuidv7.tar.gz ./packages/pglite-pg_uuidv7/release",
"wasm:copy-pgtap": "mkdir -p ./packages/pglite-pgtap/release && cp ./postgres-pglite/dist/extensions/other/pgtap.tar.gz ./packages/pglite-pgtap/release",
"wasm:copy-pg_ivm": "mkdir -p ./packages/pglite-pg_ivm/release && cp ./postgres-pglite/dist/extensions/other/pg_ivm.tar.gz ./packages/pglite-pg_ivm/release",
"wasm:copy-pg_hashids": "mkdir -p ./packages/pglite-pg_hashids/release && cp ./postgres-pglite/dist/extensions/other/pg_hashids.tar.gz ./packages/pglite-pg_hashids/release",
"wasm:copy-age": "mkdir -p ./packages/pglite-age/release && cp ./postgres-pglite/dist/extensions/other/age.tar.gz ./packages/pglite-age/release",
"wasm:copy-pg_textsearch": "mkdir -p ./packages/pglite-pg_textsearch/release && cp ./postgres-pglite/dist/extensions/other/pg_textsearch.tar.gz ./packages/pglite-pg_textsearch/release",
"wasm:copy-postgis": "mkdir -p ./packages/pglite-postgis/release && cp ./postgres-pglite/dist/extensions/other/postgis.tar.gz ./packages/pglite-postgis/release",
"wasm:copy-other_extensions": "pnpm wasm:copy-postgis; pnpm wasm:copy-pg_textsearch; pnpm wasm:copy-age; pnpm wasm:copy-pg_hashids; pnpm wasm:copy-pg_ivm; pnpm wasm:copy-pgtap; pnpm wasm:copy-pg_uuidv7; pnpm wasm:copy-pgvector",
"wasm:copy-other_extensions": "pnpm wasm:copy-postgis; pnpm wasm:copy-pg_textsearch; pnpm wasm:copy-age; pnpm wasm:copy-pg_hashids; pnpm wasm:copy-pg_ivm; pnpm wasm:copy-pgtap; pnpm wasm:copy-pg_uuidv7; pnpm wasm:copy-pgvector; pnpm wasm:copy-pgmq",
"wasm:copy-initdb": "mkdir -p ./packages/pglite/release && cp ./postgres-pglite/dist/bin/initdb.* ./packages/pglite/release",
"wasm:copy-pgdump": "mkdir -p ./packages/pglite-tools/release && cp ./postgres-pglite/dist/bin/pg_dump.* ./packages/pglite-tools/release",
"wasm:copy-pglite": "mkdir -p ./packages/pglite/release/ && cp ./postgres-pglite/dist/bin/pglite.* ./packages/pglite/release/ && cp ./postgres-pglite/dist/extensions/*.tar.gz ./packages/pglite/release/",
Expand Down
2 changes: 1 addition & 1 deletion packages/benchmark/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@
"devDependencies": {
"@types/better-sqlite3": "^7.6.11",
"ascii-table3": "^0.9.0",
"better-sqlite3": "^11.1.2",
"embedded-postgres": "15.5.1-beta.9"
},
"dependencies": {
"better-sqlite3": "^12.11.1",
"wa-sqlite": "https://github.com/rhashimoto/wa-sqlite.git#v0.9.14"
},
"peerDependencies": {
Expand Down
2 changes: 2 additions & 0 deletions packages/pglite-pgmq/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
release/*
dist
45 changes: 45 additions & 0 deletions packages/pglite-pgmq/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# @electric-sql/pglite-pgmq

[pgmq](https://github.com/pgmq/pgmq) extension for [PGlite](https://pglite.dev).

## Installation

```bash
npm install @electric-sql/pglite-pgmq
```

## Usage

```typescript
import { PGlite } from '@electric-sql/pglite'
import { pgmq } from '@electric-sql/pglite-pgmq'

const pg = new PGlite({
extensions: {
pgmq,
},
})

await pg.exec('CREATE EXTENSION IF NOT EXISTS pgmq;')

// create a queue
await pg.exec(`SELECT pgmq.create('my_queue'`);

// send a message as JSON
await pg.exec(`SELECT * from pgmq.send(
queue_name => 'my_queue',
msg => '{"foo": "bar1"}'
);`)

// read a message
const msg = await pg.exec(`SELECT * FROM pgmq.read(
queue_name => 'my_queue',
vt => 30,
qty => 2
);`)

```

## License

Apache-2.0
21 changes: 21 additions & 0 deletions packages/pglite-pgmq/eslint.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import globals from 'globals'
import rootConfig from '../../eslint.config.js'

export default [
...rootConfig,
{
ignores: ['release/**/*', 'dist/**/*'],
},
{
languageOptions: {
globals: {
...globals.browser,
...globals.node,
},
},
rules: {
...rootConfig.rules,
'@typescript-eslint/no-explicit-any': 'off',
},
},
]
64 changes: 64 additions & 0 deletions packages/pglite-pgmq/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
{
"name": "@electric-sql/pglite-pgmq",
"version": "0.0.1",
"description": "pgmq extension for PGlite",
"author": "Electric DB Limited",
"homepage": "https://pglite.dev",
"license": "Apache-2.0",
"keywords": [
"postgres",
"sql",
"database",
"wasm",
"pglite",
"pqmq"
],
"private": false,
"publishConfig": {
"access": "public"
},
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
".": {
"import": {
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
},
"require": {
"types": "./dist/index.d.cts",
"default": "./dist/index.cjs"
}
}
},
"files": [
"dist"
],
"repository": {
"type": "git",
"url": "git+https://github.com/electric-sql/pglite.git",
"directory": "packages/pglite-pgmq"
},
"scripts": {
"build": "tsup && tsx scripts/bundle-wasm.ts",
"check:exports": "attw . --pack --profile node16",
"lint": "eslint ./src ./tests --report-unused-disable-directives --max-warnings 0",
"format": "prettier --write ./src ./tests",
"typecheck": "tsc",
"stylecheck": "pnpm lint && prettier --check ./src ./tests",
"test": "vitest",
"prepublishOnly": "pnpm check:exports"
},
"devDependencies": {
"@arethetypeswrong/cli": "^0.18.1",
"@electric-sql/pglite": "workspace:*",
"@electric-sql/pglite-utils": "workspace:*",
"@types/node": "^20.16.11",
"vitest": "^2.1.2"
},
"peerDependencies": {
"@electric-sql/pglite": "workspace:*"
}
}
8 changes: 8 additions & 0 deletions packages/pglite-pgmq/scripts/bundle-wasm.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import { copyFiles, findAndReplaceInDir } from '@electric-sql/pglite-utils/scripts/fileUtils'

async function main() {
await copyFiles('./release', './dist')
await findAndReplaceInDir('./dist', /\.\.\/release\//g, './', ['.js', '.cjs'])
}

await main()
17 changes: 17 additions & 0 deletions packages/pglite-pgmq/src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import type {
Extension,
ExtensionSetupResult,
PGliteInterface,
} from '@electric-sql/pglite'

const setup = async (_pg: PGliteInterface, emscriptenOpts: any) => {
return {
emscriptenOpts,
bundlePath: new URL('../release/pgmq.tar.gz', import.meta.url),
} satisfies ExtensionSetupResult
}

export const pgmq = {
name: 'pgmq',
setup,
} satisfies Extension
Loading
Loading