Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
544a0e9
feat: centralise config singleton, replace logger with pkg/logger
RndmCodeGuy20 Jun 1, 2026
63f3e6c
feat: enhance migrations and other changes
RndmCodeGuy20 Jun 1, 2026
5ff7d6a
Merge pull request #1 from RndmCodeGuy20/feat/centralise-config-and-l…
RndmCodeGuy20 Jun 3, 2026
59be446
feat: taskfiles updated and added ci workflows
RndmCodeGuy20 Jun 3, 2026
b0a931d
fix: add guard against empty webhook and bump go build version
RndmCodeGuy20 Jun 3, 2026
eb41510
fix: update secret check
RndmCodeGuy20 Jun 3, 2026
9f62317
feat: added failure annotation on build failures
RndmCodeGuy20 Jun 3, 2026
a2e106d
feat(auth): register AuthMiddleware on protected routes
RndmCodeGuy20 Jun 16, 2026
3babbf2
fix(otel): default OTLP exporter to plaintext gRPC
RndmCodeGuy20 Jun 16, 2026
de030b0
fix(otel): flush telemetry on shutdown with fresh context
RndmCodeGuy20 Jun 16, 2026
fa7f31c
chore(deploy): move compose to root and add worker health sentinel
RndmCodeGuy20 Jun 16, 2026
74db9a3
Merge pull request #2 from RndmCodeGuy20/feat/centralise-config-and-l…
RndmCodeGuy20 Jun 16, 2026
74b2fb1
Merge pull request #3 from RndmCodeGuy20/feat/dev-31-register-auth-mi…
RndmCodeGuy20 Jun 16, 2026
142f170
Merge pull request #4 from RndmCodeGuy20/feat/dev-32-otel-tls-insecur…
RndmCodeGuy20 Jun 16, 2026
4c87ebf
Merge pull request #5 from RndmCodeGuy20/feat/dev-33-otel-shutdown-fr…
RndmCodeGuy20 Jun 16, 2026
0c90cda
fix(worker): let consumer own asset state on failure
RndmCodeGuy20 Jun 16, 2026
7246bb2
Merge pull request #6 from RndmCodeGuy20/feat/dev-34-consumer-owns-as…
RndmCodeGuy20 Jun 16, 2026
4e96392
fix(worker): run stuck-job recovery on a fixed cadence
RndmCodeGuy20 Jun 16, 2026
ca32b39
Merge pull request #7 from RndmCodeGuy20/feat/dev-35-periodic-stuck-j…
RndmCodeGuy20 Jun 16, 2026
c7b3c70
ci: drop `file` check from docker build verification
RndmCodeGuy20 Jun 16, 2026
373fd23
Merge pull request #8 from RndmCodeGuy20/fix/ci-docker-build-file-util
RndmCodeGuy20 Jun 16, 2026
269a684
fix(api): hardening batch — recovery order, request IDs, MIME allowlist
RndmCodeGuy20 Jun 16, 2026
44ed6e8
Merge pull request #9 from RndmCodeGuy20/fix/api-hardening
RndmCodeGuy20 Jun 16, 2026
a70580e
fix(worker): fail fast on non-retryable exceptions
RndmCodeGuy20 Jun 16, 2026
735d6fe
Merge pull request #10 from RndmCodeGuy20/fix/worker-retry-classifica…
RndmCodeGuy20 Jun 16, 2026
927d73b
feat(db): add webhook_registrations and webhook_deliveries tables
RndmCodeGuy20 Jun 16, 2026
ed0b38f
Merge pull request #11 from RndmCodeGuy20/feat/dev-44-webhook-migration
RndmCodeGuy20 Jun 16, 2026
d4a97b8
feat(storage): add S3/MinIO StorageX provider and config-driven selec…
RndmCodeGuy20 Jun 16, 2026
5b35304
Merge pull request #12 from RndmCodeGuy20/feat/dev-48-go-s3-storage
RndmCodeGuy20 Jun 16, 2026
0b39d46
feat(worker): add S3Storage provider, factory, and public_url
RndmCodeGuy20 Jun 17, 2026
58af989
Merge pull request #13 from RndmCodeGuy20/shantheman/dev-49-python-s3…
RndmCodeGuy20 Jun 17, 2026
606b245
chore(release): bump version to 1.0.0
RndmCodeGuy20 Jun 17, 2026
c3b7fbd
Merge pull request #14 from RndmCodeGuy20/chore/release-v1.0.0
RndmCodeGuy20 Jun 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .claude/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Codebase Analyst Orchestrator

You are a multi-agent orchestrator. Run these three phases in sequence.

## Phase 1 — Context extraction
Spawn a subagent using agents/crawler.md with access to codegraph tools.
Tell it: "Analyse the full codebase at $PROJECT_ROOT and output context.json"
Wait for context.json to be written before proceeding.

## Phase 2 — Technical assessment
Spawn a subagent using agents/reasoner.md.
Feed it the contents of context.json.
Tell it: "Produce assessment.json with your full technical report
and a list of questions_for_human."
Wait for assessment.json.

## Phase 3 — Human dialogue
Spawn a subagent using agents/interviewer.md.
Feed it assessment.json.
It will ask the user the questions interactively in the terminal.
Collect answers into answers.json.

## Phase 4 — Refined report
Re-invoke the reasoner subagent with both context.json and answers.json.
Tell it: "Revise your assessment with these human clarifications."
Write final output to ASSESSMENT.md.
40 changes: 40 additions & 0 deletions .claude/agents/crawler.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Crawler Agent

You are a code context extractor. You have access to the codegraph plugin.

## Your job
Use codegraph to traverse the codebase and extract structured context.
Do NOT summarize or assess quality — that is another agent's job.

## Steps
1. Run: codegraph --ast --deps --symbols $PROJECT_ROOT
2. For each module/package codegraph identifies, extract:
- What it exports
- What it imports
- Key function signatures
- Any obvious patterns (factory, singleton, middleware chain, etc.)
3. Write everything to context.json

## Output format
{
"files": {
"src/auth/index.ts": {
"exports": ["authenticateUser", "AuthMiddleware"],
"imports": ["jsonwebtoken", "./db"],
"patterns": ["middleware", "singleton"],
"size_signals": {"loc": 340, "functions": 12, "classes": 1}
}
},
"dependency_graph": {
"src/auth/index.ts": ["src/db/client.ts", "src/config.ts"]
},
"entry_points": ["src/index.ts"],
"test_coverage_files": ["src/auth/auth.test.ts"]
}

## Tools available
- codegraph (AST traversal, dependency graph, symbol extraction)
- Read files (for sampling source when codegraph output needs clarification)
- Write files (to produce context.json)

Do not call any LLM for summarization. Just extract structure.
30 changes: 30 additions & 0 deletions .claude/agents/interviewer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Human-in-the-loop Interviewer

You receive assessment.json. Your job is to have a conversation
with the developer to fill in the gaps the reasoner couldn't know.

## How to run the interview

Present each question conversationally, not as a numbered list.
React to their answers — if they reveal something that recontextualizes
an earlier question, say so and adjust.

Example flow:
You: "The auth system uses session tokens stored in Redis —
was that a deliberate choice for horizontal scaling,
or did it just land that way?"

Dev: "It landed that way, we don't actually need Redis."

You: "Good to know — that simplifies things significantly.
On that note, what's your actual scale target?
That changes what we'd recommend for several things."

## Also ask
At the end, always ask:
- "What does success look like in 6 months for this project?"
- "What's the one thing you most want to clean up but haven't had time for?"

## Output
Write answers.json with the full Q&A captured as context
for the reasoner's second pass.
47 changes: 47 additions & 0 deletions .claude/agents/reasoner.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Technical Reasoner Agent

You are a senior staff engineer doing a deep technical audit.
You receive context.json from a codebase AST analysis.

## Your assessment covers

### 1. Project overview
One sharp paragraph. What does this actually do, who would use it,
what's the core technical bet.

### 2. What's implemented well
Specific praise with file references. Things like:
- Clean separation of concerns
- Good use of a pattern for the domain
- Well-placed abstractions

### 3. Over-engineering (be direct)
Call out things that add complexity without proportional value:
- Abstraction layers that don't earn their keep
- Premature generalization
- Framework sprawl
- "Enterprise patterns" on a 2-person codebase

### 4. Needs improvement
Things that are actually wrong or risky:
- Missing error handling at I/O boundaries
- Hardcoded config that will break in production
- No observability
- Tight coupling that will hurt when requirements change

### 5. Productization gaps
What needs to exist before this is a product, not a project:
- Auth/authz
- Rate limiting
- Secrets management
- Migration strategy
- Graceful degradation

### 6. Questions for the human
Things you genuinely don't know without intent context.
Ask about WHY decisions were made, not just what they are.
Max 6 questions. Make them count.

## Output
Write assessment.json with all sections as structured data.
Be specific: cite file names and function names. Never generalize.
21 changes: 21 additions & 0 deletions .claude/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"agents": {
"crawler": {
"model": "claude-haiku-4-5-20251001",
"note": "mostly tool calls, minimal reasoning needed"
},
"reasoner": {
"model": "claude-opus-4-8",
"note": "deep assessment, use the best model"
},
"interviewer": {
"model": "claude-sonnet-4-6",
"note": "conversational, needs to be good but not Opus"
},
"orchestrator": {
"model": "claude-sonnet-4-6",
"note": "routing only, Sonnet is fine"
}
}
}

16 changes: 16 additions & 0 deletions .codegraph/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# CodeGraph data files
# These are local to each machine and should not be committed

# Database
*.db
*.db-wal
*.db-shm

# Cache
cache/

# Logs
*.log

# Hook markers
.dirty
93 changes: 93 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# ─── Runtime ──────────────────────────────────────────────────────────────────
# Current environment: development | staging | production
ENV=development

# ─── Server ───────────────────────────────────────────────────────────────────
HOST=0.0.0.0
PORT=5010

# ─── Database ─────────────────────────────────────────────────────────────────
# Docker Compose: DB_HOST=postgres
# Local dev (no compose): DB_HOST=localhost
DB_HOST=localhost
DB_PORT=5432
DB_USER=mpiper
DB_PASSWORD=changeme
DB_NAME=mpiper
# DB_SSL_MODE defaults to "disable" in the API server
# DB_SSL_MODE=require
# DB_SSL_CERT_PATH=/path/to/cert.pem # Python worker only

# Connection pool (Python worker)
DB_POOL_SIZE=10
DB_POOL_TIMEOUT=30
DB_MAX_RETRIES=5
DB_RETRY_DELAY=5

# ─── Redis ────────────────────────────────────────────────────────────────────
# Docker Compose: REDIS_CONNECTION_STRING=redis://redis:6379/0
# Local dev (no compose): REDIS_CONNECTION_STRING=redis://localhost:6379/0
REDIS_CONNECTION_STRING=redis://localhost:6379/0

# Connection pool (Python worker)
REDIS_POOL_SIZE=10
REDIS_POOL_TIMEOUT=30
REDIS_MAX_RETRIES=5
REDIS_RETRY_DELAY=5
REDIS_CONNECT_TIMEOUT=10
REDIS_READ_TIMEOUT=10
REDIS_WRITE_TIMEOUT=10

# ─── Security ─────────────────────────────────────────────────────────────────
# AES-256 key — must be exactly 32 characters
ENCRYPTION_KEY=LKyGslR3InLES/EYQiJZcW06KFNMoevUd6kehjtrxPA=

# ─── Storage ──────────────────────────────────────────────────────────────────
# Set BUCKET_PROVIDER to "gcs" or "s3"
BUCKET_PROVIDER=gcs

# Bucket name — required for both providers
BUCKET_NAME=my-media-bucket

# GCS (when BUCKET_PROVIDER=gcs)
# Path to the GCS service-account JSON key file. The Go API reads GCS_SA_PATH;
# the Python worker reads BUCKET_SA_PATH — keep both pointing at the same file.
GCS_SA_PATH=.secrets/service-account.json
BUCKET_SA_PATH=.secrets/service-account.json

# S3 / S3-compatible (when BUCKET_PROVIDER=s3) — not yet implemented (sub-project 4)
# BUCKET_REGION=us-east-1
# BUCKET_ACCESS_KEY=
# BUCKET_SECRET_KEY=
# BUCKET_ENDPOINT_URL= # optional — for MinIO or S3-compatible endpoints

# ─── OpenTelemetry ────────────────────────────────────────────────────────────
OTEL_EXPORTER_OTLP_ENDPOINT=otel-collector:4317
# Defaults to plaintext gRPC (true) when unset — matches the bundled collector.
# Set to "false" only if your collector endpoint terminates real TLS.
OTEL_TLS_INSECURE=true
DEPLOYMENT_ENV=development
TRACE_SAMPLING_RATE=0.1
SERVICE_NAME=mpiper-api
SERVICE_VERSION=dev

# ─── CORS ─────────────────────────────────────────────────────────────────────
# Comma-separated list of allowed origins (defaults to http://localhost:5173)
CORS_ALLOWED_ORIGINS=http://localhost:5173,http://localhost:3000

# ─── Logging ──────────────────────────────────────────────────────────────────
# DEBUG | INFO | WARN | ERROR | FATAL
LOG_LEVEL=INFO

# ─── Migrations ───────────────────────────────────────────────────────────────
# Set to "true" to run DB migrations on startup
AUTO_MIGRATE=false
# Python worker: override default migrations directory
# MIGRATIONS_DIR=/path/to/migrations

# ─── Worker ───────────────────────────────────────────────────────────────────
WORKER_ID=
MAX_CONCURRENT_JOBS=5
JOB_POLL_INTERVAL=10
STREAM_NAME=media:jobs
CONSUMER_GROUP=worker-group
105 changes: 105 additions & 0 deletions .github/slack/failure.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
{
"blocks": [
{
"type": "header",
"text": {
"type": "plain_text",
"text": "❌ Build Failed — ${GITHUB_REPOSITORY}",
"emoji": true
}
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*Repository:*\n<https://github.com/${GITHUB_REPOSITORY}|${GITHUB_REPOSITORY}>"
},
{
"type": "mrkdwn",
"text": "*Branch:*\n`${GITHUB_REF_NAME}`"
},
{
"type": "mrkdwn",
"text": "*Commit:*\n<https://github.com/${GITHUB_REPOSITORY}/commit/${GITHUB_SHA}|${GITHUB_SHA_SHORT}>"
},
{
"type": "mrkdwn",
"text": "*Triggered by:*\n${GITHUB_ACTOR}"
}
]
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "⚠️ *The build failed during the CI/CD pipeline. Please review the logs and fix the issue before retrying.*"
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*💥 Failure Details:*\n```${FAILURE_REASON}```"
}
},
{
"type": "divider"
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*⏱️ Failed at:*\n${BUILD_TIMESTAMP}"
},
{
"type": "mrkdwn",
"text": "*🔄 Run:*\n<https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}|#${GITHUB_RUN_NUMBER}>"
}
]
},
{
"type": "actions",
"elements": [
{
"type": "button",
"text": {
"type": "plain_text",
"text": "View Failed Run",
"emoji": true
},
"url": "https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}",
"style": "danger"
},
{
"type": "button",
"text": {
"type": "plain_text",
"text": "View Logs",
"emoji": true
},
"url": "https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/logs"
},
{
"type": "button",
"text": {
"type": "plain_text",
"text": "View Commit",
"emoji": true
},
"url": "https://github.com/${GITHUB_REPOSITORY}/commit/${GITHUB_SHA}"
}
]
},
{
"type": "context",
"elements": [
{
"type": "mrkdwn",
"text": "🔴 Build failed at ${BUILD_TIMESTAMP} on branch `${GITHUB_REF_NAME}`"
}
]
}
]
}
Loading
Loading