Skip to content

Commit b7e060d

Browse files
Add npm publishing and update documentation (#719)
* Changes required by Gorelase version > 2 * npm-publish script supports conditional publishing. If gorelease in not in "Release" mode, npm packages are not pushed, so we can avoid multitudes of snapshot packages being pushed to public registry * Create npm packages after goreleaser via a hook script The hook runs after. If goreaser is not in release mode, npm will build in dry-run mode, so the packages are not pushed to registry * Update documentation with NPM installation instructions * Switch npm package scope to @kosli * Fix: Add the missing bin/kosli JS Shim * Fix: Token Variable Mismatch * Fix: Silent Postinstall Failures * Update npm/README.md Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> * refactor(npm-publish): replace sed/perl with jq and harden publish script - Use jq instead of sed/perl for JSON version updates (portable across macOS and Linux, handles JSON correctly) - Separate pack and publish into two distinct phases so all packages are packed before any are published - Add npm_publish_with_retry with exponential backoff (3 attempts) - Fail fast with clear error messages on pack or publish failure * Fix: Frontmatter formatting * Consistent formatting of package.json files * Include npm packages in binary provenance processing * Add directory and engines specification to packages * Documention updates: - npx is not supported - package @kosli/cli should be used to install * Fix three issues in npm postinstall and publish script - postinstall: exit 1 on unsupported platform to match bin/kosli shim behaviour - npm-publish.sh: use process substitution instead of pipe to while loop so set -e catches failures inside the loop - npm-publish.sh: fix out-of-scope max_attempts variable in publish error message * Update scripts/npm-publish.sh Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> * Integrate npm package build and publish into GoReleaser pipeline - Copy each platform binary into its npm package dir via per-build post hooks - Run npm-publish.sh after release (dry-run on snapshots) via after hook - Clean npm bin dirs and tarballs before each build via before hooks * Update scripts/npm-publish.sh Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> * Update scripts/npm-publish.sh Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> * Update scripts/npm-publish.sh Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> * Update scripts/npm-publish.sh Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> * Add --provenance flag to npm publish when running in GitHub Actions * Fix temp file leak and add npm provenance in GitHub Actions - Clean up temp file on jq/mv failure for wrapper package.json update, consistent with the platform loop - Pass --provenance to npm publish when running in GitHub Actions * Added distribution: goreleaser-pro and GORELEASER_KEY: ${{ secrets.KOSLI_GORELEASERPRO }} — that's the standard way the goreleaser-action picks up the pro license. * Add npm installation test job to install-script-tests workflow - Test npm install -g @kosli/cli on all 6 supported platforms - Trigger on release (published) to test newly published packages - Also trigger on push/PR to npm/**, .goreleaser.yml, and scripts/npm-publish.sh * Select npm tag snapshot for now * Removed macos-13. macos-13 is the only GitHub-hosted x64 macOS runner but it's not available in all GitHub org configuration * Refine dry-run condition in npm-publish script for clarity * Add prepack scripts to fail fast when binary is missing npm pack silently succeeds even if the binary is absent, producing a broken package. Each platform package now checks for bin/kosli (or bin/kosli.exe on Windows) before packing; the wrapper also checks for install.js. * Exclude node_modules from package.json search in npm-publish script * Refactor binary validation in postinstall script to use fs.accessSync for executable check to address @dangrondahl ai review kosli version false negatives in install.js --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
1 parent b164de9 commit b7e060d

32 files changed

Lines changed: 798 additions & 5 deletions

File tree

.github/workflows/install-script-tests.yml

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,13 +7,21 @@ on:
77
- '.github/workflows/install-script-tests.yml'
88
- 'bin/test_install_script.sh'
99
- 'bin/test_install_script_over_homebrew.sh'
10+
- 'npm/**'
11+
- '.goreleaser.yml'
12+
- 'scripts/npm-publish.sh'
1013
pull_request:
1114
paths:
1215
- 'install-cli.sh'
1316
- '.github/workflows/install-script-tests.yml'
1417
- 'bin/test_install_script.sh'
1518
- 'bin/test_install_script_over_homebrew.sh'
19+
- 'npm/**'
20+
- '.goreleaser.yml'
21+
- 'scripts/npm-publish.sh'
1622
workflow_dispatch:
23+
release:
24+
types: [published]
1725

1826
jobs:
1927
test-script:
@@ -63,4 +71,34 @@ jobs:
6371
shell: bash
6472
run: |
6573
chmod +x install-cli.sh
66-
bash bin/test_install_script_over_homebrew.sh --token ${{ secrets.GITHUB_TOKEN }}
74+
bash bin/test_install_script_over_homebrew.sh --token ${{ secrets.GITHUB_TOKEN }}
75+
76+
# Note: this job installs from the public npm registry, so on push/PR
77+
# it tests the currently published version — not the code being changed.
78+
# That still catches regressions. The release trigger is what tests new releases.
79+
test-npm:
80+
name: Test npm install on ${{ matrix.os }}
81+
runs-on: ${{ matrix.os }}
82+
strategy:
83+
matrix:
84+
os:
85+
- ubuntu-latest # linux/x64
86+
- ubuntu-24.04-arm # linux/arm64
87+
- macos-latest # darwin/arm64
88+
- windows-latest # win32/x64
89+
- windows-11-arm # win32/arm64
90+
91+
steps:
92+
- name: Install @kosli/cli via npm
93+
shell: bash
94+
run: |
95+
TAG="${{ github.event.release.tag_name }}"
96+
if [[ "${{ github.event_name }}" == "release" && "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
97+
npm install -g @kosli/cli@latest
98+
else
99+
npm install -g @kosli/cli@snapshot
100+
fi
101+
102+
- name: Verify kosli binary works
103+
shell: bash
104+
run: kosli version

.github/workflows/release.yml

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -149,11 +149,17 @@ jobs:
149149
- name: Run GoReleaser
150150
uses: goreleaser/goreleaser-action@v7
151151
with:
152+
distribution: goreleaser-pro
152153
version: '~> v2' # latest
153154
args: release --clean ${{ steps.get-tag-notes.outputs.args }}
154155
env:
155156
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
156157
FURY_TOKEN: ${{ secrets.FURY_TOKEN }}
158+
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
159+
GORELEASER_KEY: ${{ secrets.KOSLI_GORELEASERPRO }}
160+
161+
- name: Copy npm packages into dist for provenance
162+
run: find npm -name "*.tgz" -exec cp {} dist/ \;
157163

158164
- uses: actions/upload-artifact@v7
159165
with:
@@ -164,7 +170,7 @@ jobs:
164170
- name: Prepare artifacts list
165171
id: prepare-artifacts-list
166172
run: |
167-
ARTIFACTS=$(jq '[reduce .[] as $item (
173+
GORELEASER_ARTIFACTS=$(jq '[reduce .[] as $item (
168174
[];
169175
if ($item.type == "Archive") then
170176
. + [{ template_name: ($item.goos + "-" + $item.goarch), path: $item.path }]
@@ -175,6 +181,18 @@ jobs:
175181
end
176182
)][]' dist/artifacts.json)
177183
184+
NPM_ARTIFACTS=$(find dist -maxdepth 1 -name "*.tgz" -printf '%f\n' \
185+
| jq -R '{
186+
template_name: ("npm-" + sub("-[0-9]+\\.[0-9]+\\.[0-9]+.*\\.tgz$"; "")),
187+
path: ("dist/" + .)
188+
}' \
189+
| jq -s '.')
190+
191+
ARTIFACTS=$(jq -n \
192+
--argjson g "$GORELEASER_ARTIFACTS" \
193+
--argjson n "$NPM_ARTIFACTS" \
194+
'$g + $n')
195+
178196
echo "artifacts<<nEOFn" >> $GITHUB_OUTPUT
179197
echo "${ARTIFACTS}" >> $GITHUB_OUTPUT
180198
echo "nEOFn" >> $GITHUB_OUTPUT

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ docs.kosli.com/resources/_gen/*
1717
docs.kosli.com/content/client_reference/kosli*
1818
docs.kosli.com/public/
1919
docs.kosli.com/.netlify
20+
npm/cli*/bin/*
21+
npm/*/kosli*.tgz
2022
*.tar.gz
2123
*~
2224
/.idea

.goreleaser.yml

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ project_name: kosli
33
before:
44
hooks:
55
- go mod tidy
6+
- rm -rf npm/cli-*/bin
7+
- find npm -name "*.tgz" -delete
68
builds:
79
- id: kosli
810
binary: kosli
@@ -27,6 +29,19 @@ builds:
2729
- goos: windows
2830
goarch: arm
2931
main: ./cmd/kosli/
32+
hooks:
33+
post:
34+
- cmd: >-
35+
bash -c '
36+
OS="{{ .Os }}";
37+
ARCH="{{ .Arch }}";
38+
[ "$OS" = "windows" ] && OS="win32";
39+
[ "$ARCH" = "amd64" ] && ARCH="x64";
40+
EXT="";
41+
[ "{{ .Os }}" = "windows" ] && EXT=".exe";
42+
mkdir -p npm/cli-${OS}-${ARCH}/bin &&
43+
cp "{{ .Path }}" npm/cli-${OS}-${ARCH}/bin/kosli${EXT} &&
44+
chmod +x npm/cli-${OS}-${ARCH}/bin/kosli${EXT}'
3045
3146
archives:
3247
-
@@ -37,11 +52,9 @@ archives:
3752
- goos: windows
3853
formats: [zip]
3954

40-
4155
# docs for nfpm can be found here: https://goreleaser.com/customization/nfpm/
4256
nfpms:
4357
- id: kosli
44-
4558
# You can change the file name of the package.
4659
#
4760
# Default:`{{ .PackageName }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}{{ with .Arm }}v{{ . }}{{ end }}{{ with .Mips }}_{{ . }}{{ end }}{{ if not (eq .Amd64 "v1") }}{{ .Amd64 }}{{ end }}`
@@ -83,6 +96,12 @@ nfpms:
8396
- src: dist/{{ .ProjectName }}_{{ .Os }}_{{ if .Amd64 }}{{ .Arch }}_v1{{ else if .Arm }}{{ .Arch }}_6{{ else if eq .Arch "arm64" }}{{ .Arch }}_v8.0{{ else }}{{ .Arch }}{{ end }}/kosli
8497
dst: /usr/local/bin/kosli
8598

99+
after:
100+
hooks:
101+
- cmd: bash scripts/npm-publish.sh "{{ .Version }}"{{ if or .IsSnapshot (not .IsRelease) }} --dry-run{{ end }}
102+
# after hooks suppresses output by default. You need to add output: true to the hook to see the script's messages.
103+
output: true
104+
86105
publishers:
87106
- name: fury.io
88107
# by specifying `packages` id here goreleaser will only use this publisher

docs.kosli.com/content/getting_started/install.md

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,18 @@ curl -L https://github.com/kosli-dev/cli/releases/download/v{{< cli-version >}}/
8787
sudo mv kosli /usr/local/bin/kosli
8888
```
8989

90+
{{< /tab >}}
91+
92+
{{< tab "NPM" >}}
93+
You can install Kosli CLI system-wide with `npm` from the default registry <https://registry.npmjs.org>
94+
95+
```shell {.command}
96+
npm install -g @kosli/cli
97+
```
98+
99+
Using `npx` is currently not supported
100+
101+
90102
{{< /tab >}}
91103

92104
{{< tab "From source" >}}
@@ -100,7 +112,6 @@ make build
100112

101113
{{< /tabs >}}
102114

103-
104115
## Verifying the installation worked
105116

106117
Run this command:

npm/README.md

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# NPM Packaging
2+
3+
This directory contains the npm package structure for distributing the Kosli CLI via npm, following the same pattern used by [esbuild](https://github.com/evanw/esbuild).
4+
5+
## Structure
6+
7+
```
8+
npm/
9+
├── wrapper/ # @kosli/cli — the package users install
10+
│ ├── bin/kosli # JS shim that detects the platform and runs the binary
11+
│ ├── install.js # postinstall script that validates the binary
12+
│ └── package.json # declares optionalDependencies for all platforms
13+
├── cli-darwin-arm64/ # @kosli/cli-darwin-arm64
14+
│ ├── bin/kosli # the native binary — see below
15+
│ └── package.json # declares os/cpu fields for platform filtering
16+
├── cli-darwin-x64/ # @kosli/cli-darwin-x64
17+
│ ├── bin/kosli # the native binary — see below
18+
│ └── package.json # declares os/cpu fields for platform filtering
19+
├── cli-linux-arm/ # @kosli/cli-linux-arm
20+
│ ├── bin/kosli # the native binary — see below
21+
│ └── package.json # declares os/cpu fields for platform filtering
22+
├── cli-linux-arm64/ # @kosli/cli-linux-arm64
23+
│ ├── bin/kosli # the native binary — see below
24+
│ └── package.json # declares os/cpu fields for platform filtering
25+
├── cli-linux-x64/ # @kosli/cli-linux-x64
26+
│ ├── bin/kosli # the native binary — see below
27+
│ └── package.json # declares os/cpu fields for platform filtering
28+
├── cli-win32-arm64/ # @kosli/cli-win32-arm64
29+
│ ├── bin/kosli.exe # the native binary — see below
30+
│ └── package.json # declares os/cpu fields for platform filtering
31+
└── cli-win32-x64/ # @kosli/cli-win32-x64
32+
├── bin/kosli.exe # the native binary — see below
33+
└── package.json # declares os/cpu fields for platform filtering
34+
```
35+
36+
## How it works
37+
38+
Users install a single package:
39+
40+
```sh
41+
npm install @kosli/cli
42+
```
43+
44+
or if using in continuous integration you can install globally:
45+
46+
```sh
47+
npm install -g @kosli/cli
48+
```
49+
50+
npm resolves the `optionalDependencies` declared in the wrapper's `package.json` and installs only the platform-specific package that matches the current OS and CPU architecture — all non-matching packages are silently skipped. The wrapper's `bin/kosli` JS shim then locates the binary inside the installed platform package and executes it.
51+
52+
> **`npx` is not supported.** `npx @kosli/cli` does not install optional dependencies, so the platform binary is never fetched and the command fails. Always install the package before running it.
53+
54+
## The `bin/` directories are populated by goreleaser
55+
56+
The platform package `bin/` directories are **not committed to git**. They are populated automatically during the release process by a post-build hook in [`.goreleaser.yml`](../.goreleaser.yml):
57+
58+
```yaml
59+
hooks:
60+
post:
61+
- cmd: >-
62+
bash -c '
63+
OS="{{ .Os }}";
64+
ARCH="{{ .Arch }}";
65+
[ "$OS" = "windows" ] && OS="win32";
66+
[ "$ARCH" = "amd64" ] && ARCH="x64";
67+
EXT="";
68+
[ "{{ .Os }}" = "windows" ] && EXT=".exe";
69+
mkdir -p npm/cli-${OS}-${ARCH}/bin &&
70+
cp "{{ .Path }}" npm/cli-${OS}-${ARCH}/bin/kosli${EXT} &&
71+
chmod +x npm/cli-${OS}-${ARCH}/bin/kosli${EXT}'
72+
```
73+
74+
This hook runs once per build target immediately after goreleaser compiles the binary. It applies the following naming conventions:
75+
76+
| goreleaser | npm package dir |
77+
|------------|-----------------|
78+
| `linux` | `linux` |
79+
| `darwin` | `darwin` |
80+
| `windows` | `win32` |
81+
| `amd64` | `x64` |
82+
| `arm64` | `arm64` |
83+
| `arm` | `arm` |
84+
85+
Windows binaries are copied as `kosli.exe`; all others as `kosli`. The `windows/arm` combination is excluded from builds.
86+
87+
The `before` hooks in `.goreleaser.yml` clean up stale artifacts before each build run:
88+
89+
```yaml
90+
before:
91+
hooks:
92+
- rm -rf npm/cli-*/bin
93+
- find npm -name "*.tgz" -delete
94+
```
95+
96+
## Publishing
97+
98+
Packages are published to the [npm public registry](https://registry.npmjs.org). Platform packages must be published before the wrapper, since the wrapper's `optionalDependencies` references them by version. After a goreleaser build has populated the `bin/` directories:
99+
100+
```sh
101+
# Publish platform packages first
102+
(cd npm/cli-linux-x64 && npm publish)
103+
(cd npm/cli-linux-arm64 && npm publish)
104+
(cd npm/cli-linux-arm && npm publish)
105+
(cd npm/cli-darwin-x64 && npm publish)
106+
(cd npm/cli-darwin-arm64 && npm publish)
107+
(cd npm/cli-win32-x64 && npm publish)
108+
(cd npm/cli-win32-arm64 && npm publish)
109+
110+
# Then publish the wrapper
111+
(cd npm/wrapper && npm publish)
112+
```
113+
114+
Each package directory contains an `.npmrc` that sets the auth token:
115+
116+
```text
117+
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
118+
```
119+
120+
## Automated Publishing with npm-publish.sh
121+
122+
The `scripts/npm-publish.sh` script automates the npm packaging and publishing process. It injects the version into all `package.json` files, packs each package into a `.tgz`, and optionally publishes them.
123+
124+
### Usage
125+
126+
```bash
127+
scripts/npm-publish.sh <version> [--dry-run]
128+
```
129+
130+
### Arguments
131+
132+
- `<version>`: Required. A SemVer string — either `X.Y.Z` (stable) or `X.Y.Z-TAG` (pre-release).
133+
- `--dry-run` (optional second argument): Pack packages but skip publishing.
134+
135+
### Behavior
136+
137+
1. Injects `<version>` into the `version` field of all `package.json` files.
138+
2. Updates the `optionalDependencies` version references in `npm/wrapper/package.json` to match.
139+
3. Runs `npm pack` on each platform package, then on the wrapper.
140+
4. Unless `--dry-run` is set, runs `npm publish --tag <tag>` on each package.
141+
142+
The dist-tag is determined by the version format:
143+
144+
| Version format | npm dist-tag |
145+
|----------------|--------------|
146+
| `X.Y.Z` | `latest` |
147+
| `X.Y.Z-*` | `snapshot` |
148+
149+
### Integration with GoReleaser
150+
151+
GoReleaser calls this script automatically via the `after` hook once all platform binaries have been built and copied into the `bin/` directories:
152+
153+
```yaml
154+
after:
155+
hooks:
156+
- cmd: bash scripts/npm-publish.sh "{{ .Version }}" ...
157+
output: true
158+
```
159+
160+
The script output is surfaced in the goreleaser log (`output: true`).
161+
162+
## Versioning
163+
164+
All packages share the same version number. When releasing, `npm-publish.sh` updates it automatically in all eight `package.json` files — the seven platform packages and the wrapper — as well as the `optionalDependencies` version pins in `npm/wrapper/package.json`. There is no need to edit these files manually.

npm/cli-darwin-arm64/.npmrc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
//registry.npmjs.org/:_authToken=${NPM_TOKEN}

npm/cli-darwin-arm64/README.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# @kosli/cli-darwin-arm64
2+
3+
This is the macOS ARM64 platform binary for the Kosli CLI (Apple Silicon). **Do not install this package directly.**
4+
5+
Install the main package instead, which selects the right binary for your platform automatically:
6+
7+
```sh
8+
npm install -g @kosli/cli
9+
```
10+
11+
See the [Kosli CLI repository](https://github.com/kosli-dev/cli) for documentation and source code.

0 commit comments

Comments
 (0)