Skip to content

Commit a9d521c

Browse files
authored
fix: updated diagrams and docs (#238)
* fix: updated diagram * fix: updated docs * fix: middleware doc update * fix: detected issues
1 parent d9f86f9 commit a9d521c

7 files changed

Lines changed: 32 additions & 22 deletions

File tree

docs/architecture/middleware.md

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,12 @@ processing in reverse order.
88

99
The middleware is applied in this order (outermost first):
1010

11-
1. **RequestSizeLimitMiddleware** - Rejects oversized requests
12-
2. **RateLimitMiddleware** - Enforces per-user/per-endpoint limits
13-
3. **CacheControlMiddleware** - Adds cache headers to responses
14-
4. **MetricsMiddleware** - Collects HTTP request metrics
11+
1. **CORSMiddleware** - Handles Cross-Origin Resource Sharing headers
12+
2. **CacheControlMiddleware** - Adds cache headers to responses
13+
3. **RequestSizeLimitMiddleware** - Rejects oversized requests
14+
4. **CSRFMiddleware** - Validates CSRF tokens on state-changing requests
15+
5. **RateLimitMiddleware** - Enforces per-user/per-endpoint limits
16+
6. **MetricsMiddleware** - Collects HTTP request metrics
1517

1618
## Request Size Limit
1719

@@ -85,6 +87,16 @@ Path templates use pattern replacement to reduce metric cardinality:
8587

8688
UUIDs, numeric IDs, and MongoDB ObjectIds are replaced with `{id}` to prevent metric explosion.
8789

90+
## CSRF Protection
91+
92+
`CSRFMiddleware` implements the **double-submit cookie** pattern. At login the server issues two cookies: an httpOnly `access_token` cookie (the JWT) and a readable `csrf_token` cookie. The CSRF token is HMAC-signed against the `access_token` so it cannot be forged independently.
93+
94+
On every mutating request (POST, PUT, DELETE, PATCH) that targets an `/api/` path, the middleware reads the `csrf_token` cookie and the `X-CSRF-Token` request header, then validates that they match (constant-time comparison) and that the token's HMAC signature is valid for the current `access_token`. Requests that fail any check receive a 403 response.
95+
96+
Safe methods (GET, HEAD, OPTIONS), auth endpoints (`/api/v1/auth/login`, `/api/v1/auth/register`), non-API paths, and unauthenticated requests (no `access_token` cookie) are exempt.
97+
98+
**Frontend behaviour:** The API interceptor in `api-interceptors.ts` auto-injects `authStore.csrfToken` into the `X-CSRF-Token` header for every non-GET request. The store obtains the token from the login response body and refreshes it by reading the `csrf_token` cookie on auth verification (`auth.svelte.ts`).
99+
88100
## System Metrics
89101

90102
In addition to HTTP metrics, the middleware module provides system-level observables:
@@ -119,4 +131,8 @@ These expose:
119131

120132
HTTP request telemetry and system-level observables
121133

134+
- :material-shield-lock:{ .lg .middle } **[csrf.py](https://github.com/HardMax71/Integr8sCode/blob/main/backend/app/core/middlewares/csrf.py)**
135+
136+
CSRF token validation for state-changing requests
137+
122138
</div>

docs/architecture/rate-limiting.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ Configuration is cached in Redis for 5 minutes to reduce database load while all
150150

151151
## Configuration
152152

153-
Rate limiting is controlled by environment variables:
153+
Rate limiting is controlled by TOML settings:
154154

155155
| Variable | Default | Description |
156156
|---------------------------|------------------|------------------------------------------------------|

docs/assets/images/system_diagram.svg

Lines changed: 1 addition & 1 deletion
Loading

docs/components/schema-manager.md

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,14 @@
11
# Schema management
22

3-
The backend manages MongoDB collection schemas — indexes, validators, and TTL policies. These are initialized at process start, whether the process is the main API or a standalone worker.
3+
The backend manages MongoDB collection schemas — indexes and TTL policies. These are initialized at process start, whether the process is the main API or a standalone worker.
44

55
Kafka event serialization is handled entirely by FastStream with Pydantic JSON; there is no schema registry involved. See [Event System Design](../architecture/event-system-design.md) for details on event serialization.
66

77
## MongoDB schema
88

9-
The `SchemaManager` class in `app/db/schema/schema_manager.py` applies idempotent, versioned migrations to MongoDB. Each migration is a short async function that creates indexes or sets collection validators. The class tracks which migrations have run in a `schema_versions` collection, storing the migration id, description, and timestamp.
9+
MongoDB indexes are defined declaratively on each Beanie `Document` subclass via an inner `Settings` class. When `init_beanie()` runs at startup, Beanie reads the `indexes` list from every registered document model and calls `create_indexes` on the collection. Because `create_indexes` is idempotent (a no-op when an index with the same name and spec already exists), every process can safely call `init_beanie()` on boot without worrying about duplicate work.
1010

11-
When `apply_all()` is called, it walks through an ordered list of migrations and skips any that already have a record in `schema_versions`. If the migration hasn't been applied, it runs the function and then marks it done. This design means you can safely call `apply_all()` on every startup without worrying about duplicate work — MongoDB's `create_indexes` is a no-op when indexes with matching names and specs already exist.
12-
13-
The system currently has nine migrations covering the main collections. The events collection gets the most attention: a unique index on `event_id`, compound indexes for queries by event type, aggregate, user, service, and status, a TTL index for automatic expiration, and a text search index across several fields. It also has a JSON schema validator set to moderate/warn mode, meaning MongoDB logs validation failures but doesn't reject writes.
14-
15-
Other migrations create indexes for user settings snapshots, replay sessions, notifications and notification rules, idempotency keys (with a one-hour TTL), sagas, execution results, and DLQ messages (with a seven-day TTL). The idempotency and DLQ TTL indexes automatically clean up old documents without manual intervention.
11+
Each document class owns its own index definitions. For example, the events collection declares a unique index on `event_id`, compound indexes for queries by event type, aggregate, user, service, and status, a TTL index for automatic expiration, and a text search index across several fields. Other documents define TTL indexes for idempotency keys (one-hour TTL) and DLQ messages (seven-day TTL) so old documents are cleaned up automatically.
1612

1713
Repositories don't create their own indexes — they only read and write. This separation keeps startup behavior predictable and prevents the same index being created from multiple code paths.
1814

@@ -22,13 +18,11 @@ During API startup, the `lifespan` function in `dishka_lifespan.py` initializes
2218

2319
## Local development
2420

25-
To force a specific MongoDB migration to run again, delete its document from `schema_versions`. To start fresh, point the app at a new database. Migrations are designed to be additive; the system doesn't support automatic rollbacks. If you need to undo a migration in production, you'll have to drop indexes or modify validators manually.
21+
Indexes are additive and idempotent. To rebuild an index with different options, drop it manually in `mongosh` and restart the process — `init_beanie()` will recreate it from the document class definition. To start fresh, point the app at a new database.
2622

2723
## Key files
2824

2925
| File | Purpose |
3026
|--------------------------------------------------------------------------------------------------------------------------------|----------------------------|
31-
| [`schema_manager.py`](https://github.com/HardMax71/Integr8sCode/blob/main/backend/app/db/schema/schema_manager.py) | MongoDB migrations |
3227
| [`typed.py`](https://github.com/HardMax71/Integr8sCode/blob/main/backend/app/domain/events/typed.py) | Domain events (Pydantic BaseModel) |
33-
| [`mappings.py`](https://github.com/HardMax71/Integr8sCode/blob/main/backend/app/infrastructure/kafka/mappings.py) | Event-to-topic routing |
3428
| [`dishka_lifespan.py`](https://github.com/HardMax71/Integr8sCode/blob/main/backend/app/core/dishka_lifespan.py) | Startup initialization |

docs/components/workers/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ For debugging a specific worker, run it directly:
4646

4747
```bash
4848
cd backend
49-
python -m workers.run_saga_orchestrator
49+
uv run python -m workers.run_saga_orchestrator
5050
```
5151

5252
## Scaling

docs/frontend/routing.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Routes are defined in `App.svelte` with authentication guards for protected page
1515
| `/settings` | `Settings.svelte` | Yes | No | User preferences |
1616
| `/notifications` | `Notifications.svelte` | Yes | No | Notification center |
1717
| `/admin/events` | `AdminEvents.svelte` | Yes | Yes | Event browser |
18+
| `/admin/executions` | `AdminExecutions.svelte` | Yes | Yes | Execution management |
1819
| `/admin/sagas` | `AdminSagas.svelte` | Yes | Yes | Saga monitoring |
1920
| `/admin/users` | `AdminUsers.svelte` | Yes | Yes | User management |
2021
| `/admin/settings` | `AdminSettings.svelte` | Yes | Yes | System settings |
@@ -105,6 +106,7 @@ Use the `route` directive for link navigation:
105106
| Component | Path | Purpose |
106107
|------------------------|-----------------------------------------|----------------------------|
107108
| `AdminEvents.svelte` | `src/routes/admin/AdminEvents.svelte` | Event browser with filters |
109+
| `AdminExecutions.svelte` | `src/routes/admin/AdminExecutions.svelte` | Execution list and priority management |
108110
| `AdminSagas.svelte` | `src/routes/admin/AdminSagas.svelte` | Saga status monitoring |
109111
| `AdminUsers.svelte` | `src/routes/admin/AdminUsers.svelte` | User CRUD and rate limits |
110112
| `AdminSettings.svelte` | `src/routes/admin/AdminSettings.svelte` | System configuration |

docs/reference/configuration.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ For production, mount `secrets.toml` from a Kubernetes Secret at `/app/secrets.t
6060
| `KUBERNETES_CA_CERTIFICATE_PATH` | Custom CA cert for K8s API | — |
6161
| `K8S_POD_CPU_LIMIT` | CPU limit per pod | `1000m` |
6262
| `K8S_POD_MEMORY_LIMIT` | Memory limit per pod | `128Mi` |
63-
| `K8S_POD_CPU_REQUEST` | CPU request (guaranteed) | `1000m` |
63+
| `K8S_POD_CPU_REQUEST` | CPU request (guaranteed) | `200m` |
6464
| `K8S_POD_MEMORY_REQUEST` | Memory request (guaranteed) | `128Mi` |
6565
| `K8S_POD_EXECUTION_TIMEOUT` | Max execution time in seconds | `300` |
6666
| `K8S_NAMESPACE` | Namespace for executor pods | `integr8scode` |
@@ -107,12 +107,10 @@ For production, mount `secrets.toml` from a Kubernetes Secret at `/app/secrets.t
107107
??? info "Legend"
108108
| Key | Description | Default |
109109
|-----|-------------|---------|
110-
| `ENABLE_TRACING` | Enable distributed tracing | `true` |
111-
| `JAEGER_AGENT_HOST` | Jaeger agent hostname | `jaeger` |
112-
| `JAEGER_AGENT_PORT` | Jaeger agent UDP port | `6831` |
110+
| `OTLP_TRACES_ENDPOINT` | OTLP gRPC endpoint; tracing is enabled when non-empty | Code default: `""` (disabled); config.toml: `http://jaeger:4317` |
113111
| `TRACING_SERVICE_NAME` | Service name in traces | `integr8scode-backend` |
114112
| `TRACING_SERVICE_VERSION` | Version in trace metadata | `1.0.0` |
115-
| `TRACING_SAMPLING_RATE` | Sample rate (0.0-1.0) | `0.1` |
113+
| `TRACING_SAMPLING_RATE` | Sample rate (0.0-1.0) | `1.0` in config.toml (`0.1` code default) |
116114

117115
## Dead Letter Queue
118116

0 commit comments

Comments
 (0)