Skip to content

Commit 7b7dfd3

Browse files
aorinevoaorinevo
committed
docs: add developer readme
1 parent 82fdf2d commit 7b7dfd3

1 file changed

Lines changed: 330 additions & 0 deletions

File tree

‎DEVELOPER.md‎

Lines changed: 330 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,330 @@
1+
# WeBWorK Developer Guide
2+
3+
This guide is for developers who want to contribute to WeBWorK or run it locally for development. For general information and end-user documentation, see [README.md](README.md) and the [WeBWorK wiki](https://webwork.maa.org/wiki/Main_Page).
4+
5+
## Tech Stack
6+
7+
| Component | Technology |
8+
|---|---|
9+
| Backend | Perl, [Mojolicious](https://mojolicious.org/) web framework |
10+
| Templates | Mojolicious Embedded Perl (`.html.ep` files) |
11+
| Frontend | JavaScript (ES6+), Bootstrap |
12+
| CSS | SCSS, PostCSS, Autoprefixer |
13+
| Database | MariaDB |
14+
| Job Queue | [Minion](https://docs.mojolicious.org/Minion) (SQLite-backed) |
15+
| Math Rendering | MathJax |
16+
| Problem Generation | [PG](https://github.com/openwebwork/pg) (separate repository) |
17+
| Deployment | Docker / Hypnotoad / systemd |
18+
19+
## Prerequisites
20+
21+
### Docker path (recommended)
22+
23+
- [Docker](https://docs.docker.com/get-docker/) and Docker Compose
24+
25+
### Native path
26+
27+
- Perl (see `DockerfileStage1` for the tested version)
28+
- MariaDB (or MySQL equivalent)
29+
- Node.js
30+
- The [pg](https://github.com/openwebwork/pg) repository cloned alongside webwork2
31+
- System packages for Perl modules (see `DockerfileStage1` for the full list)
32+
33+
## Local Development Setup (Docker)
34+
35+
1. **Copy config files:**
36+
37+
```bash
38+
cp docker-config/docker-compose.dist.yml docker-compose.yml
39+
cp docker-config/env.dist .env
40+
```
41+
42+
2. **Create the courses directory:**
43+
44+
```bash
45+
mkdir -p ../ww-docker-data/courses
46+
```
47+
48+
3. **Build and start (two-stage build, recommended):**
49+
50+
```bash
51+
docker build --tag webwork-base:forWW220 -f DockerfileStage1 .
52+
docker compose build
53+
docker compose up -d
54+
```
55+
56+
For a single-stage build, edit `docker-compose.yml` and change `dockerfile: DockerfileStage2` to `dockerfile: Dockerfile`, then run `docker compose build && docker compose up -d`.
57+
58+
4. **Mount local source for live development** (optional):
59+
60+
Uncomment this line in `docker-compose.yml` under the `app` service volumes to mount your local checkout into the container:
61+
62+
```yaml
63+
- ".:/opt/webwork/webwork2"
64+
```
65+
66+
When mounting locally, you must build the frontend assets on your host — the mount replaces the container's pre-built copies:
67+
68+
```bash
69+
cd htdocs && npm install && npm run generate-assets
70+
```
71+
72+
5. **Access WeBWorK** at http://localhost:8080/webwork2
73+
74+
The Docker entrypoint automatically creates an `admin` course with a default login:
75+
76+
- **URL:** http://localhost:8080/webwork2/admin
77+
- **Username:** `admin`
78+
- **Password:** `admin`
79+
80+
6. **Disable two-factor authentication** (recommended for local development):
81+
82+
Two-factor auth is enabled by default for all courses. To disable it, add the following to `conf/localOverrides.conf` (or mount a custom copy in Docker):
83+
84+
```perl
85+
$twoFA{enabled} = 0;
86+
```
87+
88+
### Docker commands
89+
90+
```bash
91+
docker compose logs -f app # Follow application logs
92+
docker compose down # Stop and remove containers
93+
docker compose up -d --build # Rebuild and restart
94+
```
95+
96+
## Local Development Setup (Native)
97+
98+
1. **Install Perl dependencies.** See `DockerfileStage1` for the full list of system packages and Perl modules required.
99+
100+
2. **Set up MariaDB.** Create a `webwork` database and a user with read/write access.
101+
102+
3. **Clone the PG repository** alongside webwork2:
103+
104+
```bash
105+
git clone https://github.com/openwebwork/pg.git ../pg
106+
```
107+
108+
4. **Copy and edit configuration files:**
109+
110+
```bash
111+
cp conf/site.conf.dist conf/site.conf
112+
cp conf/localOverrides.conf.dist conf/localOverrides.conf
113+
```
114+
115+
In `conf/site.conf`, set:
116+
- `$server_root_url` to `http://localhost:3000`
117+
- `$pg_dir` to the path of your `pg` checkout
118+
- `$database_password` to your MariaDB password
119+
120+
5. **Start the development server** (with hot reload):
121+
122+
```bash
123+
./bin/dev_scripts/webwork2-morbo
124+
```
125+
126+
If permissions require it, run as the server user:
127+
128+
```bash
129+
sudo -u www-data ./bin/dev_scripts/webwork2-morbo
130+
```
131+
132+
6. **Start the job queue worker** (in a separate terminal):
133+
134+
```bash
135+
./bin/webwork2 minion worker
136+
```
137+
138+
Note: the Minion worker does not hot reload. Restart it manually after changing task modules.
139+
140+
7. **Create the admin course:**
141+
142+
```bash
143+
bin/addcourse admin --db-layout=sql_single \
144+
--users=courses.dist/adminClasslist.lst \
145+
--professors=admin
146+
```
147+
148+
This creates the `admin` course with a default user `admin` (password: `admin`).
149+
150+
8. **Disable two-factor authentication** (recommended for local development):
151+
152+
Add the following to `conf/localOverrides.conf`:
153+
154+
```perl
155+
$twoFA{enabled} = 0;
156+
```
157+
158+
9. **Access WeBWorK** at http://localhost:3000/webwork2
159+
160+
## Project Structure
161+
162+
```
163+
webwork2/
164+
├── bin/ # CLI scripts and executables
165+
│ └── dev_scripts/ # Development-only scripts (morbo, etc.)
166+
├── lib/ # Core Perl source code
167+
│ ├── Mojolicious/ # Mojolicious app and plugins
168+
│ └── WeBWorK/ # WeBWorK business logic
169+
│ ├── ContentGenerator/ # Page controllers (one per page type)
170+
│ ├── Authen/ # Authentication modules
171+
│ ├── DB/ # Database layer (Schema, Record, Utils)
172+
│ └── ...
173+
├── templates/ # Mojolicious .html.ep templates
174+
├── htdocs/ # Frontend assets (JS, CSS, images, themes)
175+
│ ├── js/ # JavaScript modules organized by feature
176+
│ ├── css/ # Compiled CSS
177+
│ ├── themes/ # UI themes (math4, math4-red, etc.)
178+
│ └── package.json # Frontend dependencies and build scripts
179+
├── conf/ # Configuration files (.dist templates)
180+
├── assets/ # Static assets (LaTeX themes, stop words)
181+
├── courses.dist/ # Sample course directory structure
182+
├── docker-config/ # Docker configuration and entrypoint
183+
├── doc/ # License files
184+
├── logs/ # Application logs
185+
└── tmp/ # Temporary files
186+
```
187+
188+
## Architecture Overview
189+
190+
### Application entry point
191+
192+
The Mojolicious app is defined in `lib/Mojolicious/WeBWorK.pm` and started via `bin/webwork2`:
193+
194+
```bash
195+
./bin/webwork2 daemon # Start in development mode
196+
./bin/webwork2 prefork # Start in production mode (hypnotoad)
197+
```
198+
199+
### ContentGenerator pattern
200+
201+
Each page type has a corresponding Perl module in `lib/WeBWorK/ContentGenerator/`. These modules handle routing, authorization, and rendering for their respective pages. Examples: `Grades.pm`, `ProblemSets.pm`, `CourseAdmin.pm`, `Instructor/UserList.pm`.
202+
203+
### Database layer
204+
205+
The DB layer has three tiers:
206+
207+
- **`lib/WeBWorK/DB.pm`** — Top-level API for all database operations
208+
- **`lib/WeBWorK/DB/Schema/`** — Schema definitions and query builders
209+
- **`lib/WeBWorK/DB/Record/`** — Data record objects (user, set, problem, etc.)
210+
211+
### PG integration
212+
213+
The Problem Generation system lives in the separate [pg](https://github.com/openwebwork/pg) repository. It is loaded at runtime from the path configured in `$pg_dir`.
214+
215+
## Frontend Development
216+
217+
Frontend assets live in `htdocs/`. To work on JavaScript or CSS:
218+
219+
```bash
220+
cd htdocs
221+
npm install
222+
npm run generate-assets
223+
```
224+
225+
See `htdocs/package.json` for the full list of frontend dependencies.
226+
227+
Themes are located in `htdocs/themes/`.
228+
229+
## Configuration
230+
231+
WeBWorK uses a `.dist` file convention: files ending in `.dist` are templates that should be copied (without the `.dist` suffix) and customized. Never modify `.dist` files directly — your changes will be lost on upgrade.
232+
233+
**Config load order:**
234+
235+
1. `conf/site.conf` — Server-specific settings (URL, DB credentials, PG path)
236+
2. `conf/defaults.config` — Default values for all options (**do not modify**)
237+
3. `conf/localOverrides.conf` — Your customizations, overrides values from `defaults.config`
238+
239+
Optional authentication configs (LTI, LDAP, CAS, SAML2, Shibboleth) can be included from `localOverrides.conf`.
240+
241+
See [conf/README.md](conf/README.md) for full configuration and deployment documentation.
242+
243+
## Code Style and Linting
244+
245+
Formatting is enforced by CI on every pull request.
246+
247+
### Perl
248+
249+
Configured via `.perltidyrc`:
250+
251+
- Line width: 120 characters
252+
- Indentation: tabs (4-space equivalent)
253+
- Cuddled else blocks
254+
255+
Format Perl files with:
256+
257+
```bash
258+
perltidy -pro=.perltidyrc <file>
259+
```
260+
261+
### JavaScript / CSS / HTML
262+
263+
Configured via `.prettierrc`:
264+
265+
- Line width: 120 characters
266+
- Single quotes, no trailing commas
267+
- Indentation: tabs
268+
269+
```bash
270+
cd htdocs
271+
npm run prettier-check # Check formatting
272+
npm run prettier-format # Auto-fix formatting
273+
```
274+
275+
### Editor config
276+
277+
The `.editorconfig` file provides consistent settings across editors (UTF-8, LF line endings, tab indentation).
278+
279+
## Useful Scripts
280+
281+
| Script | Description |
282+
|---|---|
283+
| `bin/webwork2` | Main application entry point (Mojolicious commands) |
284+
| `bin/dev_scripts/webwork2-morbo` | Development server with hot reload |
285+
| `bin/wwsh` | WeBWorK interactive shell |
286+
| `bin/addcourse` | Create a new course |
287+
| `bin/delcourse` | Delete a course |
288+
| `bin/addadmin` | Add an admin user |
289+
| `bin/OPL-update` | Update the Open Problem Library |
290+
| `bin/check_modules.pl` | Verify Perl module dependencies |
291+
| `bin/importClassList.pl` | Import a class roster |
292+
293+
## Contributing
294+
295+
1. Fork the repository and create a feature branch from `develop`.
296+
2. Follow the code style guidelines above — CI will check formatting automatically.
297+
3. Open a pull request against `develop`. The `main` branch is reserved for hotfix pull requests only.
298+
4. For discussion or questions, use [GitHub Discussions](https://github.com/openwebwork/webwork2/discussions).
299+
300+
For more developer resources, see the [WeBWorK developer wiki](https://webwork.maa.org/wiki/Category:Developers).
301+
302+
## Troubleshooting
303+
304+
### CSS/JS not loading when mounting local source
305+
306+
When you mount `.:/opt/webwork/webwork2` in Docker, your local files replace the container's pre-built assets. Build them on your host:
307+
308+
```bash
309+
cd htdocs && npm install && npm run generate-assets
310+
```
311+
312+
Verify that `htdocs/static-assets.json` was created — this is the asset manifest the app uses to resolve hashed filenames. If the file is missing, the app cannot find the compiled CSS/JS and pages will appear unstyled.
313+
314+
**Node.js version note:** On Node 22+, a fix was applied to `htdocs/generate-assets.js` to ensure the chokidar `ready` event fires correctly and `static-assets.json` is written.
315+
316+
### Container exits with `cp: cannot stat '*.json': No such file or directory`
317+
318+
The OPL volume has a stale state — the SQL dump exists but JSON metadata files are missing. Remove the volume and let Docker recreate it:
319+
320+
```bash
321+
docker compose down
322+
docker volume rm webwork2_oplVolume
323+
docker compose up -d
324+
```
325+
326+
The first startup after this will be slower as it re-clones the Open Problem Library.
327+
328+
### Two-factor authentication prompt blocking login
329+
330+
Add `$twoFA{enabled} = 0;` to `conf/localOverrides.conf` and restart the app. See step 6 in the Docker setup above.

0 commit comments

Comments
 (0)