Skip to content

Commit 4cb4753

Browse files
matmenclaude
andcommitted
benchmark against other image libraries
bench/ compares ImageScript to ImageScript 1, Jimp, image-js, Photon, sharp and @napi-rs/canvas: decoding, encoding, image operations, text and SVG rendering, animated GIFs, encoded file sizes and install sizes. Every library runs in its own process (run.js), install sizes are measured with npm in an empty project (installs.js), report.js draws the charts and writes bench/README.md. The release workflow runs it for every regular release against the released version and commits the results; bench.yml can also be run by hand. Inputs are from Wikimedia Commons (CC BY 2.0, CC0, public domain, see bench/inputs/SOURCES.md). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
1 parent 6e8730e commit 4cb4753

24 files changed

Lines changed: 3417 additions & 1 deletion

‎.github/workflows/bench.yml‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
name: bench
2+
3+
# runs the library comparison in bench/ and commits the results (results.json, installs.json, charts, README) to the
4+
# default branch. release.yml runs it after a regular release reached npm, measuring the released version; can also
5+
# be run by hand from the Actions tab, measuring the version bench/package.json has (or the one given)
6+
on:
7+
workflow_call:
8+
inputs:
9+
version:
10+
description: ImageScript version to measure (release.yml passes the released one)
11+
type: string
12+
default: ''
13+
workflow_dispatch:
14+
inputs:
15+
version:
16+
description: ImageScript version to measure (empty keeps bench/package.json's)
17+
type: string
18+
default: ''
19+
20+
permissions:
21+
contents: write
22+
23+
concurrency:
24+
group: bench
25+
cancel-in-progress: false
26+
27+
jobs:
28+
bench:
29+
runs-on: ubuntu-latest
30+
defaults:
31+
run:
32+
working-directory: bench
33+
steps:
34+
- uses: actions/checkout@v7
35+
with:
36+
ref: ${{ github.event.repository.default_branch }}
37+
- uses: actions/setup-node@v7
38+
with:
39+
node-version: 24.x
40+
41+
- run: npm ci --no-audit --no-fund
42+
# a version that was just published can take a moment to show up on the registry
43+
- if: inputs.version != ''
44+
run: |
45+
for attempt in 1 2 3 4 5; do
46+
npm install --save-exact --no-audit --no-fund "imagescript@$VERSION" && exit 0
47+
sleep 30
48+
done
49+
exit 1
50+
env:
51+
VERSION: ${{ inputs.version }}
52+
53+
- run: node run.js
54+
- run: node installs.js
55+
- run: node report.js
56+
57+
# pushes with GITHUB_TOKEN don't trigger other workflows; rebases onto anything pushed meanwhile
58+
- run: |
59+
git add -A .
60+
if git diff --cached --quiet; then echo "nothing changed"; exit 0; fi
61+
git config user.name "github-actions[bot]"
62+
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
63+
git commit -m "benchmark ImageScript $(node -p "require('./node_modules/imagescript/package.json').version")"
64+
for attempt in 1 2 3; do
65+
git push && exit 0
66+
git pull --rebase
67+
done
68+
exit 1

‎.github/workflows/release.yml‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@ name: release
33
# publishing a GitHub release with the tag v<version> publishes that version to npm and JSR. The tag is the version:
44
# the release is checked (semver, changelog, not yet on npm/JSR), tested on every supported platform, published with
55
# the tag's version to npm (with provenance, through trusted publishing) and JSR (through GitHub OIDC, no tokens),
6-
# then the docs are deployed and, for regular releases, the version is committed to the default branch
6+
# then the docs are deployed and, for regular releases, the version is committed to the default branch and the
7+
# benchmark in bench/ is run against the release
78
on:
89
release:
910
types: [published]
@@ -110,3 +111,14 @@ jobs:
110111
git push
111112
env:
112113
VERSION: ${{ needs.check.outputs.version }}
114+
115+
# measures the release against the other libraries and commits the results to bench/ on the default branch (after
116+
# the version commit, so the two don't race)
117+
bench:
118+
needs: [check, version]
119+
if: needs.check.outputs.dist-tag == 'latest'
120+
permissions:
121+
contents: write
122+
uses: ./.github/workflows/bench.yml
123+
with:
124+
version: ${{ needs.check.outputs.version }}

‎README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,13 @@ see the [changelog](https://github.com/matmen/ImageScript/blob/master/CHANGELOG.
8585

8686
---
8787

88+
### Benchmarks
89+
90+
[bench/](bench/README.md) compares ImageScript to Jimp, image-js, Photon, sharp and @napi-rs/canvas: speed of decoding,
91+
encoding, image operations and rendering, encoded file sizes and install sizes. It runs for every release.
92+
93+
---
94+
8895
### Credits
8996

9097
The deflate/inflate implementation is derived from [fflate](https://github.com/101arrowz/fflate) and the CRC-32 from

‎bench/README.md‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# ImageScript benchmark
2+
3+
How [ImageScript](https://github.com/matmen/ImageScript) compares to other image libraries for Node.js: speed, file
4+
sizes and what installing each takes. ImageScript is plain JavaScript with no dependencies, so it runs wherever
5+
JavaScript does; sharp and @napi-rs/canvas wrap native libraries (libvips, Skia), Photon and ImageScript 1 are
6+
WebAssembly.
7+
8+
<img src="charts/install.svg" alt="Install size of each library">
9+
10+
<img src="charts/codecs.svg" alt="Decoding and encoding speed">
11+
12+
<img src="charts/operations.svg" alt="Speed of image operations">
13+
14+
<img src="charts/rendering.svg" alt="Text and SVG rendering speed">
15+
16+
<img src="charts/animation.svg" alt="Animated GIF decoding and encoding speed">
17+
18+
<img src="charts/sizes.svg" alt="Encoded file sizes">
19+
20+
## Numbers
21+
22+
Operations per second (more is faster), measured 2026-10-15 with Node.js v24.11.1 on Intel(R) Core(TM) i3-8109U CPU @ 3.00GHz, ImageScript 3.1.0.
23+
24+
| | **ImageScript 3** | **ImageScript 1** | **Jimp** | **image-js** | **Photon** | **sharp** | **@napi-rs/canvas** |
25+
|---|--:|--:|--:|--:|--:|--:|--:|
26+
| decode JPEG | **9.06** | 3.97 | 4.34 | 5.17 | 16.1 | 37.3 | 42.2 |
27+
| decode PNG | **20.4** | 20.6 | 12.2 | 7.46 | 22.0 | 43.7 | 48.1 |
28+
| decode WebP (lossy) | **7.73** | – | – | – | 7.75 | 30.3 | 53.3 |
29+
| encode JPEG (quality 80) | **9.51** | 14.6 | 1.86 | 6.76 | 6.12 | 49.0 | 30.8 |
30+
| encode PNG | **6.67** | 3.17 | 3.33 | 2.37 | 7.57 | 6.74 | 2.00 |
31+
| encode WebP (lossless) | **0.67** | – | – | – | 17.0 | 0.52 | 1.85 |
32+
| resize to half | **14.9** | 249 | 42.5 | 7.09 | 9.57 | 50.6 | 364 |
33+
| rotate by 30° | **6.39** | 4.22 | 1.20 | 0.85 | 2.64 | 10.6 | 31.7 |
34+
| blur (radius 5) | **4.32** | – | 0.01 | 1.00 | 2.54 | 23.7 | 33.5 |
35+
| composite an overlay | **272** | 408 | 22.4 | 103 | 38.3 | 95.3 | 236 |
36+
| crop | **313** | 428 | 34.4 | 46.3 | 41.1 | 928 | 934 |
37+
| clone | **596** | 575 | 700 | 563 | 42.2 | 712 | 273 |
38+
| render text | **774** | 50.1 | – | – | – | 1,896 | 3,164 |
39+
| render SVG | **3.86** | fails | – | – | – | 12.0 | 51.2 |
40+
| decode animated GIF | **4.95** | 6.77 | – | – | – | 12.5 | – |
41+
| encode animated GIF | **3.39** | 1.01 | – | – | – | 0.80 | – |
42+
43+
Encoded sizes:
44+
45+
| | **ImageScript 3** | **ImageScript 1** | **Jimp** | **image-js** | **Photon** | **sharp** | **@napi-rs/canvas** |
46+
|---|--:|--:|--:|--:|--:|--:|--:|
47+
| photo PNG | 2.2 MB | 3.0 MB | 2.7 MB | 2.8 MB | 2.8 MB | 2.8 MB | 2.1 MB |
48+
| photo JPEG | 217 kB | 257 kB | 260 kB | 260 kB | 257 kB | 218 kB | 218 kB |
49+
| photo WebP | 1.4 MB | – | – | – | 1.7 MB | 1.2 MB | 1.4 MB |
50+
| graphic PNG | 1.1 MB | 802 kB | 2.0 MB | 736 kB | 2.2 MB | 718 kB | 1.0 MB |
51+
| graphic JPEG | 275 kB | 314 kB | 315 kB | 315 kB | 314 kB | 275 kB | 276 kB |
52+
| graphic WebP | 579 kB | – | – | – | 856 kB | 239 kB | 631 kB |
53+
| animation GIF | 1.3 MB | 1.3 MB | – | – | – | 720 kB | – |
54+
55+
| | **ImageScript 3** | **ImageScript 1** | **Jimp** | **image-js** | **Photon** | **sharp** | **@napi-rs/canvas** |
56+
|---|---|---|---|---|---|---|---|
57+
| kind | JavaScript | WebAssembly | JavaScript | JavaScript | WebAssembly | native (libvips) | native (Skia) |
58+
| package | `imagescript@3.1.0` | `imagescript@1.3.1` | `jimp@1.6.1` | `image-js@1.7.1` | `@silvia-odwyer/photon-node@0.3.4` | `sharp@0.35.5` | `@napi-rs/canvas@1.0.11` |
59+
| install size | 639 kB | 6.7 MB | 26.7 MB | 47.8 MB | 2.3 MB | 30.2 MB | 35.0 MB |
60+
| packages | 1 | 1 | 64 | 55 | 1 | 9 | 2 |
61+
62+
## Method
63+
64+
- Inputs (in `inputs/`, sources in [inputs/SOURCES.md](inputs/SOURCES.md)): a 1920×1080 photo (JPEG, and made from it
65+
when the benchmark starts, PNG and lossy WebP), a 1434×1434 graphic with transparency (PNG), a 445×445 diagram with
66+
transparency placed on the photo (PNG), a 300×300 GIF with 101 frames.
67+
- Rendering: "The quick brown fox jumps over the lazy dog, 0123456789" in Source Sans 3 at 64px, and `tests/svgs/features.svg` (gradients, clipping,
68+
CSS) at 1920 pixels wide. Jimp and Photon can't render TrueType/OpenType fonts or SVGs; ImageScript 1 crashes
69+
on this SVG; @napi-rs/canvas (Skia's SVG module) ignores its CSS, so it draws a little less.
70+
- Every operation starts from the same decoded image and returns a new one: libraries that change images in place
71+
clone first, sharp runs a pipeline from raw pixels back to raw pixels, @napi-rs/canvas draws into a new canvas.
72+
- Where the library lets you choose, resizing is bilinear (ImageScript 1 only has nearest neighbor), blurring is
73+
gaussian with a radius or sigma of 5, rotating grows the image to fit. Each library uses its own defaults
74+
otherwise, e.g. its PNG compression level.
75+
- [tinybench](https://github.com/tinylibs/tinybench), at least 1.5 seconds and 3 runs per measurement, every library
76+
in its own process.
77+
- The release workflow runs the benchmark for every release and commits the results, on a GitHub Actions runner;
78+
runners vary, so compare libraries within one run rather than between runs.
79+
- Install sizes: `npm install <library>` in an empty project, the apparent size of `node_modules`.
80+
81+
Run it: `npm install` in this directory, then `node run.js` (on an idle machine; `IMAGESCRIPT=local` measures
82+
this checkout after `npm run build` instead of the version from npm), `node installs.js` and `node report.js`.

‎bench/bench.js‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
// Measures one library (node --expose-gc bench.js <name>) and prints its results as JSON: operations per second for
2+
// decoding, encoding, the image operations and rendering, and the encoded file sizes. run.js runs every library in
3+
// its own process, so one library's memory can't affect the next one's measurements
4+
import fs from 'node:fs';
5+
import sharp from 'sharp';
6+
import { Bench } from 'tinybench';
7+
import { libraries } from './libs.js';
8+
9+
const lib = libraries.find(l => l.name === process.argv[2]);
10+
if (!lib) throw new Error(`unknown library ${process.argv[2]}, one of: ${libraries.map(l => l.name).join(', ')}`);
11+
12+
const files = {
13+
'photo.jpg': fs.readFileSync(new URL('./inputs/photo.jpg', import.meta.url)),
14+
'graphic.png': fs.readFileSync(new URL('./inputs/graphic.png', import.meta.url)),
15+
'overlay.png': fs.readFileSync(new URL('./inputs/overlay.png', import.meta.url)),
16+
'animation.gif': fs.readFileSync(new URL('./inputs/animation.gif', import.meta.url))
17+
};
18+
files['photo.png'] = await sharp(files['photo.jpg']).png().toBuffer();
19+
files['photo.webp'] = await sharp(files['photo.jpg']).webp({ quality: 80 }).toBuffer();
20+
21+
const speed = {}, sizes = {};
22+
const formatName = format => ({ jpeg: 'JPEG', png: 'PNG', webp: 'WebP' })[format];
23+
24+
// collect: canvases hold their pixels in native memory that V8 doesn't count and only free it in finalizers that run
25+
// on a later event-loop turn, which tinybench's loop never yields to. So between iterations (outside the measured
26+
// time) garbage is collected and the loop gets a turn, instead of the pixels piling up until the process is killed
27+
async function measure(task, fn) {
28+
const bench = new Bench({ time: 1500, iterations: 3, warmupTime: 300, warmupIterations: 1 });
29+
bench.add(task, fn, lib.collect && globalThis.gc ? { afterEach: async () => { globalThis.gc(); await new Promise(setImmediate); } } : {});
30+
await bench.run();
31+
const r = bench.tasks[0].result;
32+
if (r.error) throw r.error;
33+
const ms = r.latency?.mean ?? r.mean;
34+
speed[task] = { opsPerSecond: Math.round(1000 / ms * 100) / 100, ms: Math.round(ms * 100) / 100, samples: r.latency?.samples?.length ?? r.samples?.length };
35+
console.error(`${task.padEnd(16)} ${lib.name.padEnd(16)} ${(1000 / ms).toFixed(2).padStart(10)} ops/s ${ms.toFixed(2).padStart(9)} ms`);
36+
}
37+
38+
const decode = data => Promise.resolve().then(() => lib.decode(data));
39+
const photo = await decode(files['photo.jpg']), overlay = await decode(files['overlay.png']);
40+
41+
for (const [task, file] of [['decode JPEG', 'photo.jpg'], ['decode PNG', 'photo.png'], ['decode WebP', 'photo.webp']])
42+
if (await decode(files[file]).then(() => true, () => false)) await measure(task, () => decode(files[file]));
43+
for (const [format, encode] of Object.entries(lib.encode))
44+
await measure(`encode ${formatName(format)}`, () => encode(photo));
45+
for (const op of ['resize', 'rotate', 'blur', 'composite', 'crop', 'clone'])
46+
if (lib[op]) await measure(op, () => lib[op](photo, overlay));
47+
// a library that has the feature but fails on this input (ImageScript 1 on the SVG) is recorded as failing
48+
for (const op of ['text', 'svg']) {
49+
if (!lib[op]) continue;
50+
const error = await Promise.resolve().then(() => lib[op]()).then(() => null, e => e);
51+
if (error) speed[op] = { error: String(error.message ?? error) };
52+
else await measure(op, () => lib[op]());
53+
}
54+
if (lib.animation) {
55+
const animation = await lib.animation.decode(files['animation.gif']);
56+
await measure('decode GIF', () => lib.animation.decode(files['animation.gif']));
57+
await measure('encode GIF', () => lib.animation.encode(animation));
58+
}
59+
60+
// encoded sizes, with each library's default settings (JPEG at quality 80, WebP lossless)
61+
for (const [name, file] of [['photo', 'photo.jpg'], ['graphic', 'graphic.png']]) {
62+
const image = await decode(files[file]);
63+
for (const [format, encode] of Object.entries(lib.encode)) sizes[`${name} ${formatName(format)}`] = (await encode(image)).length;
64+
}
65+
if (lib.animation) sizes['animation GIF'] = (await lib.animation.encode(await lib.animation.decode(files['animation.gif']))).length;
66+
67+
console.log(JSON.stringify({ speed, sizes }));

‎bench/charts/animation.svg‎

Lines changed: 15 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)