emscripten: add Docker-based one-stop build and polish demo page#6385
emscripten: add Docker-based one-stop build and polish demo page#6385ChrisJefferson wants to merge 2 commits into
Conversation
|
That is great! I have a small change request. Could you let the startup_manifest.json include the manual files, as requested in #6269 (comment)? You could do it by typing the following command in gap wasm website to record it in startup_manifest.json: or use external tools to add paths of all .six files. |
|
ping @ChrisJefferson |
|
@ChrisJefferson was AI used to assist in creating this PR? If so it would be good to acknowledge it (and perhaps the AI can be instructed to head our |
There was a problem hiding this comment.
Hmm... the commit message states the following, as if it was a benefit:
startup_manifest.json captured from a real GAP run and committed.
Removes: build_startup_manifest.js
But to me this seems like a step backwards: startup_manifest.json is a generated file, so shouldn't be in the repo. Worse, though, the code for updating it was removed. Surely the list of files in there will evolve over time?
There was a problem hiding this comment.
The code to regenerate it was very fragile, and I had trouble making it work. The new code works fine, but requires you cut+paste it into a browser javascript window.
There was a problem hiding this comment.
I think the new startup_manifest.json generation script is integrated into gap-fs.js and the web-template files. I tested this by serving my prebuilt WebAssembly GAP with the new web-templates via serve.py and following the new README. This is the startup_manifest.json I get: startup_manifest.json. It has no .log file and includes the .six manuals.
I just realized that my version of WebAssembly GAP is trying to request these non-existent manual files:
pkg/patternclass/doc/manual.six
pkg/edim/doc/manual.six
pkg/smallantimagmas/doc/manual.six. If there are no big changes to build.sh, this could still be an issue. I haven't figured out why this happened.
The coi-serviceworker.js README notes that it works on both localhost and static hosting via HTTPS. I verified this locally. So the blocks in serve.py that adds the COOP and COEP header are harmless but not needed.
There was a problem hiding this comment.
I think I've now improved this (let me know what you think)
There was a problem hiding this comment.
That is great. Thanks for the update.
|
I am thinking about enhancing gap-fs.js to support loading custom scripts via a specified path. Suppose the main files are hosted on a CDN such as GitHub or jsDelivr; this would allow users to embed a gap shell directly on their websites to run a custom script. This could be great for running an interactive demo for gap code or packages. |
|
Cleaned up the code a bit, but I thought I'd tell a little story about versions (this discussion was writen by AI, but proof-read by me, by going through the long process of trying to get things updated). Why this stays on emscripten 3.1.23The old Dockerfile pinned 1. A real GASMAN bug (fixed here)Built against a modern emsdk (I tried 5.0.7), GAP links and starts, then aborts during library loading — GASMAN scans the stack conservatively, and on wasm reaches register-resident pointers via Fix is one line in 2. The actual blocker: the terminalWith GAP itself happy on 5.0.7, the REPL double-echoes every line: GAP's line editor echoes each char (thinks it's raw) while xterm-pty's line discipline also echoes on newline (still cooked). I instrumented
Bumping xterm-pty doesn't help — the versions line up wrong:
The worker integration was dropped at 0.10.0, and even 0.12 only claims 3.1.47. There is no xterm-pty that keeps our integration and targets a modern emscripten; moving forward means rewriting the terminal front end (new API, xterm 4→5) onto an untested combination, for little gain on a pinned build-time toolchain producing a static site. DecisionStay on emsdk 3.1.23 + xterm-pty 0.9.4 — the only known-good pairing — but now with an accurate Dockerfile explanation, and keep the GASMAN fix (correct on any toolchain). Also here (toolchain-independent build fixes)
AI disclosureClaude Code (Opus 4.8) diagnosed both issues and made the edits; credited as co-author per |
Rework the emscripten build into a reproducible flow inside a pinned container: etc/emscripten/build-in-docker.sh builds the image, runs the wasm build, and assembles a self-contained web-example/ that can be copied to any static host. Replaces the previous mix of shell, Ruby and Node helpers. The toolchain is pinned to emsdk 3.1.23 because both halves of the REPL are sensitive to the exact version: GASMAN's conservative GC, and xterm-pty's emscriptenHack() patching emscripten's TTY device so tcsetattr() reaches the line discipline. 3.1.23 is the version that works with the pinned xterm-pty 0.9.4; a newer emsdk leaves tcsetattr unhonoured and double- echoes typed input. See the Dockerfile for the full rationale. Build robustness fixes, independent of the toolchain version: - configure zlib with --static (newer wasm-ld rejects its shared .so test targets; GAP only needs libz.a); - remove the generated src/c_*.c and ffdata.* before the native build so reruns regenerate them instead of failing; - assemble-website.sh copies the data tree resiliently (some packages ship dangling .dSYM symlinks that aborted the copy and left lib/, grp/ uncopied, causing startup 404s), drops pkg/log/ test logs, ships gap.worker.js only if the build produced one, and asserts lib/init.g landed in the output. Also fix the outdated documentation links in the demo page and refresh startup_manifest.json. AI assistance: Claude Code (Opus 4.8) was used to make these changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GASMAN's emscripten_scan_stack/emscripten_scan_registers calls -- which spill pointers held in wasm registers so the conservative collector can see them -- were guarded by `#ifdef EMSCRIPTEN`. Older emscripten defined the bare `EMSCRIPTEN` macro, so the block compiled and worked; modern emscripten removed it in favour of `__EMSCRIPTEN__`, which silently disabled the block. Without the register scan, the fallback stack scan misses roots that live only in registers, so live bags are freed and the heap is corrupted (observed as "index out of bounds" / "corrupted its heap memory area" aborts on a recent emsdk). Update the guard to the modern macro. AI assistance: Claude Code (Opus 4.8) diagnosed and made this change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ChrisJefferson
force-pushed
the
emscripten-polish
branch
from
ff18398 to
a5bacfe
Compare
3 participants
Reworks the emscripten build to be reproducible inside a pinned emsdk:3.1.23 container, replacing the previous mix of shell, Ruby, and Node helpers. New entry point: etc/emscripten/build-in-docker.sh builds the image, runs the wasm build inside it, and assembles a self-contained web-example/ directory copyable to any static host.
This doesn't change functionality, but it (hopefully) makes it clearer what is going on, makes the created page nicer, and cleans up the docker script. You can now just run
etc/emscripten/build-in-docker.shand get a full finished build ready to deploy.