Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions GUIDELINES_NEST_REFERENCE_APP.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# GUIDELINES_NEST_REFERENCE_APP.md

## Core Philosophy - This app SERVES the eight nest-native libraries
## Core Philosophy - This app SERVES the nine nest-native libraries

This is `nest-native/reference-app`: a production-shaped, multi-tenant
work-tracking SaaS whose only job is to put **all eight
work-tracking SaaS whose only job is to put **all nine
[nest-native](https://github.com/nest-native) libraries** — `drizzle`, `trpc`,
`messaging`, `kafka`, `jobs`, `asyncapi`, `ai-sdk`, and `lockout` — under
`messaging`, `kafka`, `jobs`, `asyncapi`, `ai-sdk`, `lockout`, and `cache` — under
realistic backend pressure and show how they compose. It is **read first and
run second**. Every decision optimizes for a reader who wants to lift a pattern
into their own app.
Expand All @@ -15,10 +15,10 @@ app's constitution.

### 1. What This App Is (and Is Not)

- It **serves the libraries**. Every module exists to exercise one of the eight
- It **serves the libraries**. Every module exists to exercise one of the nine
under realistic pressure — persistence, a typed API, reliable domain events,
an event backbone, deferred jobs, a documented event catalog, a streaming AI
assistant, and brute-force login lockout.
assistant, brute-force login lockout, and tag-invalidated read caching.
- It is **not** a product, **not** a starter or scaffold, and **not** a generic
NestJS boilerplate. There is no CLI, no second frontend, no multi-database
story, no GraphQL — those are deliberate omissions, not gaps to fill.
Expand Down
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
<p>
<a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License" /></a>
<img src="https://img.shields.io/badge/node-%3E%3D22-brightgreen.svg" alt="Node version" />
<img src="https://img.shields.io/badge/libraries-8%2F8-0f766e.svg" alt="All eight nest-native libraries" />
<img src="https://img.shields.io/badge/libraries-9%2F9-0f766e.svg" alt="All nine nest-native libraries" />
</p>

A production-shaped **multi-tenant work-tracking SaaS** — think a small Linear/Asana — that puts **all eight [nest-native](https://github.com/nest-native) libraries** under realistic backend pressure and composes them the way a real product would: persistence, a typed API, reliable domain events, an event backbone, deferred background jobs, a documented event catalog, a streaming AI assistant, and brute-force login lockout.
A production-shaped **multi-tenant work-tracking SaaS** — think a small Linear/Asana — that puts **all nine [nest-native](https://github.com/nest-native) libraries** under realistic backend pressure and composes them the way a real product would: persistence, a typed API, reliable domain events, an event backbone, deferred background jobs, a documented event catalog, a streaming AI assistant, brute-force login lockout, and tag-invalidated read caching.

It is not a product. It **serves the libraries** — a credible demo a team could adapt, and a feedback loop: if something feels awkward here, that's signal to add API in a separate PR to the relevant library (that's how, for example, `MessagingModule.forRoot`'s `imports` option and `KafkaInboxConsumer`'s `dedupKey` argument came to exist).

Expand All @@ -21,7 +21,7 @@ Follow one journey through the code and every library shows up where a real syst
5. **The event contracts are published.** An **AsyncAPI 3.0 catalog** at `/asyncapi` documents every event so another team could subscribe to your streams.
6. **AI reads the activity.** A streaming **project assistant** turns a project's recent activity into a status update, token by token.

## Eight libraries, eight chapters
## Nine libraries, nine chapters

| Library | Its job in the story | Where in the code |
| --- | --- | --- |
Expand All @@ -32,6 +32,7 @@ Follow one journey through the code and every library shows up where a real syst
| [`@nest-native/jobs`](https://github.com/nest-native/jobs) | **Deferred work** — the assignment reminder: enqueued in the same transaction as the `task.assigned` projection (`uniqueKey` = the event's dedup key), executed exactly once by the worker | `src/modules/reminders/`, `TaskAssignedProjection` in `src/modules/activity/`, `src/database/schema/jobs.ts` |
| [`@nest-native/asyncapi`](https://github.com/nest-native/asyncapi) | **The event catalog** — an AsyncAPI 3.0 doc so other teams integrate with your streams | `src/modules/events-catalog/`, served at `/asyncapi` from `src/main.ts` |
| [`@nest-native/ai-sdk`](https://github.com/nest-native/ai-sdk) | **AI over your data** — a streaming assistant that summarizes a project's activity | `src/modules/assistant/` (`@AiStream`), `POST /projects/:projectId/assistant` |
| [`@nest-native/cache`](https://github.com/nest-native/cache) | **Read caching without Redis** — expensive reads (`projects.*`, `activity.list`) cached in an in-memory L1 with **tag-based invalidation**: mutations evict by tag (`project:42`), the activity projection invalidates its own feed at the write site, and every entry's TTL is only the backstop. Coherence travels through `@stalefree/core`'s bus seam (in-process by default; set `CACHE_SOCKET_PATH` for the app + worker split) | `src/cache/cache.setup.ts` (`CacheModule.forRootAsync`), `projects.router.ts` + `activity.router.ts` (`wrap` + tags), `activity.service.ts` (write-site invalidation) |
| [`@nest-native/lockout`](https://github.com/nest-native/lockout) | **Brute-force defense** — failed logins lock an identity (by email OR source IP) *before* the credential is checked; a Drizzle-backed counter on the same SQLite DB, `Retry-After` on the lock, reset on success. tRPC has no HTTP body for the guard's extractor to read, so the login handler calls `LockoutService` directly — the library's documented non-HTTP-transport recipe | `src/auth/lockout.setup.ts` (`LockoutModule.forRootAsync`), `src/auth/auth.service.ts` (`check` → `reportFailure`/`reportSuccess`) |

Underneath, [`@nestjs-cls/transactional`](https://www.npmjs.com/package/@nestjs-cls/transactional) (with the official Drizzle adapter) ties the outbox write to the business write — its `transactionMode: 'auto'` runs the same `@Transactional()` code against better-sqlite3's synchronous driver locally and an async driver in production.
Expand Down Expand Up @@ -92,6 +93,7 @@ src/
config/env.ts loadEnv() — single source of truth (incl. the optional kafka block)
database/ DrizzleModule wiring + schema (orgs/users/projects/tasks/activity/...) + migrations
auth/ scrypt passwords, HS256 JWT, AuthGuard, middleware; @nest-native/lockout login lockout (lockout.setup.ts)
cache/ @nest-native/cache read caching — tag invalidation through @stalefree/core (cache.setup.ts)
context/ request-scoped CURRENT_USER / CURRENT_ORGANIZATION
modules/
organizations/ users/ projects/ repos + services + tRPC routers (tenant-scoped)
Expand All @@ -117,7 +119,7 @@ npm run test:cov # with c8 coverage
npm run ci # typecheck, lint, complexity (≤15), test:cov, security:audit, build
```

Coverage here is **pragmatic, not 100%** — the 100% bar belongs to the libraries. The transactional workflow, the outbox worker, the inbox dedup, the reminder job's exactly-once scheduling and execution, the AsyncAPI catalog, the AI stream, and the login-lockout gate (fail N times → 429, even the right password is refused while locked) all have explicit tests. CI runs on **Node 22**.
Coverage here is **pragmatic, not 100%** — the 100% bar belongs to the libraries. The transactional workflow, the outbox worker, the inbox dedup, the reminder job's exactly-once scheduling and execution, the AsyncAPI catalog, the AI stream, and the login-lockout gate (fail N times → 429, even the right password is refused while locked), and cache coherence (mutations refresh cached reads long before TTL — tag invalidation, not expiry) all have explicit tests. CI runs on **Node 22**.

Two **optional, local-only** layers sit on top (neither runs in CI, and forks
work without them):
Expand All @@ -143,6 +145,7 @@ ritual, and agent instructions — in
| `@nest-native/drizzle` | `0.3.x` · `@nest-native/trpc` `0.6.x` · `@nest-native/kafka` `0.2.x` |
| `@nest-native/messaging` | `0.2.x` · `@nest-native/asyncapi` `0.1.x` · `@nest-native/ai-sdk` `0.4.x` (on `ai@7`) |
| `@nest-native/lockout` | `0.3.x` (on `@authlock/core` `0.3.x`) |
| `@nest-native/cache` | `0.1.x` (on `@stalefree/core` `0.1.x`) |
| Drizzle ORM | `0.45.x` · `@nestjs-cls/transactional` `^3` |

## Philosophy
Expand Down
37 changes: 37 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
"@authlock/core": "^0.3.0",
"@nest-native/ai-sdk": "^0.5.0",
"@nest-native/asyncapi": "^0.2.0",
"@nest-native/cache": "^0.1.0",
"@nest-native/drizzle": "^0.4.0",
"@nest-native/jobs": "^0.1.0",
"@nest-native/kafka": "^0.3.0",
Expand All @@ -58,6 +59,7 @@
"@nestjs/common": "11.1.28",
"@nestjs/core": "11.1.28",
"@nestjs/platform-express": "11.1.28",
"@stalefree/core": "^0.1.0",
"@trpc/server": "11.18.0",
"ai": "^7.0.22",
"better-sqlite3": "12.11.1",
Expand Down
2 changes: 2 additions & 0 deletions src/app.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import { ProjectsModule } from './modules/projects/projects.module';
import { TasksModule } from './modules/tasks/tasks.module';
import { UsersModule } from './modules/users/users.module';
import { AppTrpcModule } from './trpc/trpc.module';
import { AppCacheModule } from './cache/cache.setup';

// Reliable-messaging wiring, now built on `@nest-native/messaging`. Kafka is an
// opt-in profile: with KAFKA_BROKERS unset, the engine's claimer publishes
Expand Down Expand Up @@ -97,6 +98,7 @@ function messagingImports(): NonNullable<ModuleMetadata['imports']> {
@Module({
imports: [
DatabaseModule,
AppCacheModule,
ClsModule.forRoot({
global: true,
plugins: [
Expand Down
82 changes: 82 additions & 0 deletions src/cache/cache.setup.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
import {
Inject,
Injectable,
Logger,
Module,
type OnApplicationShutdown,
} from '@nestjs/common';
import { CacheModule } from '@nest-native/cache';
import type { InvalidationBus } from '@stalefree/core';
import { SocketInvalidationBus } from '@stalefree/core/socket';
import { loadEnv } from '../config/env';

/**
* Wires @stalefree/core into the app: an in-memory L1 read cache with
* tag-based invalidation. TTL (CACHE_TTL_MS, default 30s) is the delivery
* backstop — a missed invalidation can only serve stale until the TTL, never
* forever.
*
* Coherence: in the DEFAULT profile the whole app is one process, so the
* cache is exactly coherent with no bus at all. When the outbox/jobs worker
* runs as a SEPARATE process (`npm run start:worker`) it writes activity rows
* the app has cached — set CACHE_SOCKET_PATH (the same value in both
* processes) and the socket bus carries invalidations across: the worker's
* projection writes evict the app's L1. No L2 store: the database is a local
* SQLite file, so a shared warm tier would just duplicate it.
*/
export const CACHE_BUS = Symbol.for('reference-app:cache-bus');

const logger = new Logger('Cache');

@Injectable()
class CacheBusLifecycle implements OnApplicationShutdown {
constructor(
@Inject(CACHE_BUS) private readonly bus: InvalidationBus | undefined,
) {}

async onApplicationShutdown(): Promise<void> {
await this.bus?.close?.();
}
}

@Module({
providers: [
{
provide: CACHE_BUS,
useFactory: async (): Promise<InvalidationBus | undefined> => {
const env = loadEnv();
if (!env.cacheSocketPath) {
return undefined; // single-process profile: L1 is coherent as-is
}
const bus = new SocketInvalidationBus({
path: env.cacheSocketPath,
onError: (error) => logger.error('cache bus error', error),
});
await bus.start();
return bus;
},
},
CacheBusLifecycle,
],
exports: [CACHE_BUS],
})
class CacheBusModule {}

@Module({
imports: [
CacheModule.forRootAsync({
imports: [CacheBusModule],
inject: [CACHE_BUS],
useFactory: (bus: InvalidationBus | undefined) => {
const env = loadEnv();
return {
defaultTtlMs: env.cacheTtlMs,
bus,
onError: (error, context) =>
logger.error(`cache error (${context})`, error),
};
},
}),
],
})
export class AppCacheModule {}
6 changes: 6 additions & 0 deletions src/config/env.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ export interface AppEnv {
lockoutLimit: number;
/** Login lockout: how long a locked identity stays locked, in ms. */
lockoutCooloffMs: number;
/** Read cache: TTL for cached reads, in ms (the delivery backstop). */
cacheTtlMs: number;
/** Read cache: unix-socket path for cross-process invalidation (app + worker). Unset = in-process only. */
cacheSocketPath: string | undefined;
outbox: {
pollIntervalMs: number;
batchSize: number;
Expand Down Expand Up @@ -131,6 +135,8 @@ export function loadEnv(): AppEnv {
authTtlSeconds: Number.parseInt(process.env.AUTH_TTL_SECONDS ?? '3600', 10),
lockoutLimit: readIntFromEnv('LOCKOUT_LIMIT', 5),
lockoutCooloffMs: readIntFromEnv('LOCKOUT_COOLOFF_MS', 15 * 60_000),
cacheTtlMs: readIntFromEnv('CACHE_TTL_MS', 30_000),
cacheSocketPath: process.env.CACHE_SOCKET_PATH,
outbox: {
pollIntervalMs: readIntFromEnv('OUTBOX_POLL_MS', 2_000),
batchSize: readIntFromEnv('OUTBOX_BATCH_SIZE', 32),
Expand Down
16 changes: 12 additions & 4 deletions src/modules/activity/activity.router.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { Inject, NotFoundException, UseGuards } from '@nestjs/common';
import { Input, Query, Router } from '@nest-native/trpc';
import { CacheService } from '@nest-native/cache';
import { z } from 'zod';
import { AuthGuard } from '../../auth/auth.guard';
import type { CurrentOrganizationContext } from '../../auth/auth-context';
Expand Down Expand Up @@ -35,17 +36,24 @@ const ListActivityInputSchema = z.object({
export class ActivityRouter {
constructor(
@Inject(ActivityService) private readonly service: ActivityService,
@Inject(CacheService) private readonly cache: CacheService,
@Inject(CURRENT_ORGANIZATION)
private readonly currentOrg: CurrentOrganizationContext | null,
) {}

@Query({ input: ListActivityInputSchema, output: z.array(ActivityEventSchema) })
list(@Input('projectId') projectId: number) {
async list(@Input('projectId') projectId: number) {
if (!this.currentOrg) {
throw new NotFoundException('No active organization for this session');
}
return this.service
.list(this.currentOrg.id, projectId)
.map((event) => ({ ...event, createdAt: new Date(event.createdAt) }));
// The RAW rows are what gets cached; the Date mapping runs per response
// (mapping before caching would hand every hit the same mapped objects —
// cached values must be treated as immutable).
const rows = await this.cache.wrap(
`org:${this.currentOrg.id}:project:${projectId}:activity`,
() => this.service.list(this.currentOrg!.id, projectId),
{ tags: [`project:${projectId}:activity`] },
);
return rows.map((event) => ({ ...event, createdAt: new Date(event.createdAt) }));
}
}
20 changes: 17 additions & 3 deletions src/modules/activity/activity.service.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { Injectable } from '@nestjs/common';
import { Inject, Injectable } from '@nestjs/common';
import { InjectTransaction } from '@nestjs-cls/transactional';
import { CacheService } from '@nest-native/cache';
import { and, desc, eq } from 'drizzle-orm';
import type { AppDatabase } from '../../database/database';
import { type ActivityEvent, activityEvents } from '../../database/schema';
Expand All @@ -23,10 +24,13 @@ export interface RecordActivityInput {
*/
@Injectable()
export class ActivityService {
constructor(@InjectTransaction() private readonly db: AppDatabase) {}
constructor(
@InjectTransaction() private readonly db: AppDatabase,
@Inject(CacheService) private readonly cache: CacheService,
) {}

record(input: RecordActivityInput): ActivityEvent | undefined {
return this.db
const row = this.db
.insert(activityEvents)
.values({
orgId: input.orgId,
Expand All @@ -42,6 +46,16 @@ export class ActivityService {
})
.returning()
.get();
if (row && row.projectId !== null) {
// Evict the project's cached feed everywhere a row was actually written
// (duplicates are no-ops and evict nothing). Fire-and-forget: `record`
// stays synchronous for the better-sqlite3 transaction it runs inside,
// and the sync driver means no other request can interleave mid-tx, so
// the pre-commit timing is not observable here; on an async driver this
// would move to an after-commit hook, with the TTL as the backstop.
void this.cache.invalidateTags([`project:${row.projectId}:activity`]);
}
return row;
}

list(orgId: number, projectId: number): ActivityEvent[] {
Expand Down
Loading