|
| 1 | +# Copilot Instructions for Defacto2 Server |
| 2 | + |
| 3 | +## Project Overview |
| 4 | + |
| 5 | +**Defacto2** is a self-contained web server written in Go that serves an archive of software history, demos, and artifacts from the scene. It's a full-stack application combining: |
| 6 | +- Backend: Go web server using Echo framework |
| 7 | +- Frontend: HTML/CSS/JavaScript templates with HTMX for interactivity |
| 8 | +- Database: PostgreSQL (optional but required for full functionality) |
| 9 | +- Asset Pipeline: Node.js-based asset compilation for CSS/JS |
| 10 | + |
| 11 | +The server is configured entirely through environment variables and includes optional support for emulation, file downloads, and image previews. |
| 12 | + |
| 13 | +## Build and Run Commands |
| 14 | + |
| 15 | +### Using Task (Recommended) |
| 16 | + |
| 17 | +This project uses [Task](https://taskfile.dev/) as the task runner. All commands are defined in `Taskfile.dist.yaml`. |
| 18 | + |
| 19 | +**Common commands:** |
| 20 | +```bash |
| 21 | +task serve # Run dev server with live reload (main command) |
| 22 | +task serve-prod # Run prod mode with live reload |
| 23 | +task test # Run full test suite |
| 24 | +task testr # Run full test suite with race detection |
| 25 | +task lint # Format and lint all code |
| 26 | +task binary # Build standalone binary |
| 27 | +``` |
| 28 | + |
| 29 | +**First time setup (REQUIRED):** |
| 30 | +```bash |
| 31 | +task _init # Install dev tool dependencies (installs goreleaser, sqlboiler, etc.) |
| 32 | +task ver # Verify all tools are installed |
| 33 | +``` |
| 34 | + |
| 35 | +**Other useful tasks:** |
| 36 | +```bash |
| 37 | +task assets # Rebuild CSS/JS assets |
| 38 | +task nil # Run nil dereference static analysis |
| 39 | +task spell # Check markdown spelling in docs |
| 40 | +task doc # Browse package documentation at localhost:8090 |
| 41 | +task pkg-update # Update all dependencies |
| 42 | +task pkg-patch # Patch dependencies to latest patch versions |
| 43 | +task build-snapshot # Build release binaries |
| 44 | +task serve-fix # Run server with database fix flag |
| 45 | +``` |
| 46 | + |
| 47 | +### Without Task |
| 48 | + |
| 49 | +If Task is not available: |
| 50 | +```bash |
| 51 | +go run server.go # Run the server |
| 52 | +go test -count 1 ./... # Run all tests |
| 53 | +go test -count 1 -race ./... # Run with race detection |
| 54 | +go run -modfile=runner/go.mod runner/runner.go # Rebuild assets |
| 55 | +``` |
| 56 | + |
| 57 | +### Running Single Tests |
| 58 | + |
| 59 | +Go test package paths are based on directory structure: |
| 60 | +```bash |
| 61 | +go test -count 1 ./handler/app # Test single package |
| 62 | +go test -count 1 ./handler/app -run TestAppNew # Run specific test |
| 63 | +go test -count 1 -v ./handler/app # Verbose output |
| 64 | +``` |
| 65 | + |
| 66 | +### Environment Configuration |
| 67 | + |
| 68 | +The Taskfile can load environment variables from `init/.env.local` for local development: |
| 69 | +```bash |
| 70 | +# Copy template to local config |
| 71 | +cp init/example.env.local init/.env.local |
| 72 | + |
| 73 | +# Edit to set D2_DATABASE_URL, D2_PROD_MODE, etc. |
| 74 | +nano init/.env.local |
| 75 | +``` |
| 76 | + |
| 77 | +This allows running `task serve-dev` with custom paths for downloads, thumbnails, and database connection without modifying the system environment. |
| 78 | + |
| 79 | +## Project Architecture |
| 80 | + |
| 81 | +### High-Level Structure |
| 82 | + |
| 83 | +``` |
| 84 | +server.go # Entry point, sets up config, database, and router |
| 85 | +├─ handler/ # HTTP route handlers (the "view" layer) |
| 86 | +│ ├─ app/ # Main website routes (file records, artist pages, etc.) |
| 87 | +│ ├─ html3/ # HTML/ASCII art text file rendering |
| 88 | +│ ├─ download/ # File download endpoint |
| 89 | +│ ├─ cache/ # Cache management handlers |
| 90 | +│ ├─ sess/ # Session/authentication handlers |
| 91 | +│ └─ [others]/ # Specialized handlers (demozoo, pouet, CSDB sync, etc.) |
| 92 | +├─ model/ # Database models (generated by sqlboiler ORM) |
| 93 | +├─ internal/ # Internal packages |
| 94 | +│ ├─ command/ # CLI command logic (address, config, fix) |
| 95 | +│ ├─ config/ # Environment variable parsing |
| 96 | +│ ├─ postgres/ # Database initialization and migrations |
| 97 | +│ ├─ logs/ # Structured logging setup (slog) |
| 98 | +│ └─ [others]/ # Utilities (dir paths, tag parsing, panic recovery) |
| 99 | +└─ runner/ # Separate Go module for asset build process |
| 100 | + └─ runner.go # SCSS→CSS and JS minification/bundling |
| 101 | +``` |
| 102 | + |
| 103 | +### Key Design Patterns |
| 104 | + |
| 105 | +1. **Echo Framework Routes**: Handler functions follow Echo's signature pattern: |
| 106 | + ```go |
| 107 | + func (app *App) RouteName(c echo.Context) error |
| 108 | + ``` |
| 109 | + Handlers parse requests, query the database, and render templates. |
| 110 | + |
| 111 | +2. **SQLBoiler ORM**: Database models in `model/` are generated from the PostgreSQL schema. Raw SQL queries use the generated query builders. |
| 112 | + |
| 113 | +3. **Template Rendering**: All HTML is rendered server-side using Go's `html/template` package. Templates are embedded in the binary. |
| 114 | + |
| 115 | +4. **Embed FS**: Static files (CSS, JS, views) are embedded in the binary at compile time using `//go:embed` directives. |
| 116 | + |
| 117 | +5. **Environment-Based Configuration**: No config files—all settings via `D2_*` environment variables parsed in `internal/config/`. |
| 118 | + |
| 119 | +6. **Global Cache**: The `app.Cache` struct holds site-wide cached data (record counts, etc.) to avoid repeated database queries. |
| 120 | + |
| 121 | +### Database Schema |
| 122 | + |
| 123 | +The primary table is `files`, which stores metadata about scene artifacts. It has relationships to: |
| 124 | +- Releasers and sceners (artists/groups) |
| 125 | +- Platforms and file types |
| 126 | +- File content hashes and locations |
| 127 | + |
| 128 | +Database path must be provided via `D2_DATABASE_URL` environment variable. The `/init/` directory contains Postgres initialization scripts and the sqlboiler configuration. |
| 129 | + |
| 130 | +### Asset Build Pipeline |
| 131 | + |
| 132 | +Assets are processed separately via the `runner` module: |
| 133 | +- **Input**: `assets/css/*.css` and `assets/js/*.js` |
| 134 | +- **Process**: SCSS compilation, minification, bundling |
| 135 | +- **Output**: Built files written to embedded filesystem |
| 136 | +- **Trigger**: Runs before `binary` and `serve` tasks |
| 137 | +
|
| 138 | +The `runner.go` file orchestrates this process using Hugo and related Go tools. |
| 139 | +
|
| 140 | +## Key Conventions |
| 141 | +
|
| 142 | +### Code Organization |
| 143 | +
|
| 144 | +1. **Handler packages**: Each route handler is in its own subdirectory under `handler/`. Shared logic goes in `handler/app/internal/`. |
| 145 | +
|
| 146 | +2. **Model usage**: Import models from `github.com/Defacto2/server/model`. Use sqlboiler's query builders rather than raw SQL when possible. See `docs/patterns.md` for detailed SQLBoiler examples. |
| 147 | +
|
| 148 | +3. **Error handling**: Return errors from handlers; Echo middleware converts them to HTTP responses. Custom error types are defined in `handler/app/error.go`. |
| 149 | +
|
| 150 | +4. **Testing**: Test files live alongside code with `_test.go` suffix. Use `nalgeon/be` assertion library (see imports for examples). |
| 151 | +
|
| 152 | +5. **Template data**: Handlers populate a context struct and pass it to templates. Template helpers are methods on the `App` type in `handler/app/app.go`. |
| 153 | +
|
| 154 | +### Database |
| 155 | +
|
| 156 | +The application uses **PostgreSQL** (not MySQL) with [SQLBoiler](https://github.com/aarondl/sqlboiler) ORM for type-safe database queries. Key database setup: |
| 157 | +- Connection via `D2_DATABASE_URL` environment variable (required for full functionality) |
| 158 | +- Models auto-generated in `model/` from schema |
| 159 | +- Soft-delete support: `WithDeleted()` query mod includes deleted records |
| 160 | +- See `docs/database.md` for troubleshooting and migration info from MySQL |
| 161 | +
|
| 162 | +#### Database Schema Overview |
| 163 | +
|
| 164 | +**Primary Table: `files`** (50+ columns) |
| 165 | +The core table storing scene artifacts, releases, demos, and artwork. Auto-generated Go struct in `internal/postgres/models/files.go`. |
| 166 | +
|
| 167 | +**Key Columns:** |
| 168 | +- **Identity**: `id` (primary key), `uuid` (global identifier) |
| 169 | +- **Relationships**: `group_brand_for` (primary releaser), `group_brand_by` (secondary releaser) |
| 170 | +- **Metadata**: `record_title`, `filename`, `filesize`, `platform`, `section` (category), `comment` |
| 171 | +- **Dates**: `date_issued_year/month/day` (published), `createdat`, `updatedat`, `deletedat` (soft-delete), `deletedby` |
| 172 | +- **Credits**: `credit_text`, `credit_program`, `credit_illustration`, `credit_audio` (comma-separated scener names) |
| 173 | +- **External IDs**: `web_id_demozoo`, `web_id_pouet`, `web_id_16colors`, `web_id_github`, `web_id_youtube` |
| 174 | +- **Archive Content**: `file_zip_content` (list of filenames in archive), `file_magic_type` (MIME type) |
| 175 | +- **Integrity**: `file_integrity_strong` (SHA384 hash), `file_security_alert_url` |
| 176 | +- **Emulation**: `dosee_run_program`, `dosee_hardware_cpu/graphic/audio`, `dosee_*` flags for DOS game configuration |
| 177 | +- **Rendering**: `retrotxt_readme` (text file to display), `list_links`, `list_relations` |
| 178 | +
|
| 179 | +**Related Entities** (accessed through `files` columns): |
| 180 | +- **Releasers/Groups**: Parsed from `group_brand_for` and `group_brand_by` fields. Use `model.Releaser` for querying distinct releasers and their stats. |
| 181 | +- **Sceners**: Parsed from credit columns. Use `model.Scener` to query all files credited to a scener. |
| 182 | +
|
| 183 | +**Soft-Delete Pattern:** |
| 184 | +- Deleted records have `deletedat` set to a timestamp and `deletedby` containing the UUID of who deleted it |
| 185 | +- Queries exclude soft-deleted by default; include them with `qm.WithDeleted()` |
| 186 | +- Epoch year for dates is 1980; year/month/day can be null for unknown dates |
| 187 | +
|
| 188 | +See `docs/patterns.md` for SQLBoiler query examples and `docs/database.md` for schema migration details. |
| 189 | +
|
| 190 | +### Error Types |
| 191 | +
|
| 192 | +Common errors defined in `handler/app/app.go`: |
| 193 | +- `ErrValue`, `ErrNegative`, `ErrSession` - User input validation |
| 194 | +- `ErrTmpl` - Template rendering failure |
| 195 | +- `ErrCorrupt` - Cache data issues |
| 196 | +- Use these or wrap with `fmt.Errorf()` for context |
| 197 | +
|
| 198 | +### Logging |
| 199 | +
|
| 200 | +Uses Go's structured logging (`slog`) initialized in `internal/logs/`. Log levels are configured via `D2_LOG_LEVEL` env var. |
| 201 | +
|
| 202 | +### Embedded Files |
| 203 | +
|
| 204 | +- Public web files: `//go:embed public/**/*` |
| 205 | +- View templates: `//go:embed view/**/*` |
| 206 | +- Brand text: `//go:embed public/text/defacto2.txt` |
| 207 | + |
| 208 | +Files are accessed via the embedded `FS` objects; ensure paths match the embed pattern. |
| 209 | + |
| 210 | +### Naming Conventions |
| 211 | + |
| 212 | +- **Handlers**: Method names match route purposes (e.g., `App.File`, `App.Artists`) |
| 213 | +- **Templates**: Live in `view/` directory, organized by feature |
| 214 | +- **Database queries**: Use `model` package functions (generated by sqlboiler) |
| 215 | +- **Environment variables**: Prefixed with `D2_` (e.g., `D2_PROD_MODE`, `D2_DATABASE_URL`) |
| 216 | + |
| 217 | +### Go Version |
| 218 | + |
| 219 | +Current version: **Go 1.25.5**. See `docs/patterns.md` for modern Go 1.24+ patterns and idioms available for use. |
| 220 | + |
| 221 | +## Important Dependencies |
| 222 | + |
| 223 | +- **Echo** - Web framework and HTTP router |
| 224 | +- **sqlboiler** - ORM code generator for type-safe database queries (requires running `task _init` to install) |
| 225 | +- **PostgreSQL driver** (pgx) - Database connectivity |
| 226 | +- **pnpm** - Package manager for Node.js dev dependencies (linting/formatting) |
| 227 | +- **Task** - Task runner (use `task` command, not Make) |
| 228 | +- **golangci-lint** - Linter aggregator (installed via `task _init`) |
| 229 | +- **goreleaser** - Release automation (installed via `task _init`) |
| 230 | +- **air** - Live reload tool for development (installed via `task _init`) |
| 231 | + |
| 232 | +## Testing Notes |
| 233 | + |
| 234 | +- Tests use the `be` assertion library (simple, zero-dependency testing) |
| 235 | +- Many tests mock database calls to avoid requiring a live database |
| 236 | +- Run with `-count 1` to disable test result caching (recommended for TDD) |
| 237 | +- Use `-race` flag to detect concurrency issues in network code |
| 238 | + |
| 239 | +## Common Tasks |
| 240 | + |
| 241 | +| Task | Command | |
| 242 | +|------|---------| |
| 243 | +| Start dev server | `task serve` | |
| 244 | +| Run all tests | `task test` | |
| 245 | +| Run tests with race detection | `task testr` | |
| 246 | +| Lint and format | `task lint` | |
| 247 | +| Add Go dependency | `go get -u package.path` then `go mod tidy` | |
| 248 | +| Add Node.js dev tool | `pnpm add -D package-name` | |
| 249 | +| Generate new DB models | Edit schema, run `task _init` (installs sqlboiler) then regenerate | |
| 250 | +| View API docs | `task doc` (opens localhost:8090) | |
| 251 | +| Query database examples | See `docs/patterns.md` for SQLBoiler query patterns | |
| 252 | + |
| 253 | +## Additional Resources |
| 254 | + |
| 255 | +- **`docs/`** - Architecture and development guides |
| 256 | + - `patterns.md` - Go language patterns and SQLBoiler ORM examples |
| 257 | + - `database.md` - PostgreSQL setup, troubleshooting, and migration info |
| 258 | + - `source.md` - Source setup and Task runner details |
| 259 | +- **`Taskfile.dist.yaml`** - Complete list of all available tasks with descriptions |
| 260 | +- **`init/`** - Configuration templates, linting rules, and build config |
| 261 | + |
0 commit comments