Skip to content

Commit 2ac84ac

Browse files
claudeaparcar
authored andcommitted
refactor: Separate prepare and build into independent microservices
Convert to true microservices architecture where prepare and build services have COMPLETELY SEPARATE CODEBASES with NO shared code. Services communicate ONLY via HTTP. ``` / ├── asu/ # Build service (heavy, existing codebase) │ └── Calls prepare service via HTTP └── asu-prepare/ # Prepare service (NEW, separate codebase) ├── main.py ├── build_request.py ├── package_changes.py ├── package_resolution.py ├── config.py └── Containerfile ``` NEW completely independent microservice for package resolution: **Files Created:** - `asu-prepare/main.py` - Standalone FastAPI app (170 lines) - `asu-prepare/build_request.py` - Data models (175 lines) - `asu-prepare/package_changes.py` - Migration logic (copied, 198 lines) - `asu-prepare/package_resolution.py` - Resolution (copied, 195 lines) - `asu-prepare/config.py` - Minimal config (35 lines) - `asu-prepare/pyproject.toml` - Minimal dependencies - `asu-prepare/Containerfile` - Lightweight container - `asu-prepare/README.md` - Documentation **Dependencies:** ONLY FastAPI, Pydantic, Uvicorn **NO Dependencies on:** Redis, RQ, Podman, ImageBuilder, asu/* **API:** - `POST /api/v1/prepare` - Package resolution - `GET /health` - Health check **Resources:** - 512MB RAM, 0.5 CPU - <1s response time - Completely stateless Modified to call prepare service via HTTP: **Modified Files:** - `asu/routers/api.py`: - `/build/prepare` now proxies to prepare service via HTTP - Removed services imports - Restored original build logic - Added httpx for HTTP calls - `asu/config.py`: - Added `prepare_service_url` setting - Default: `http://asu-prepare:8001` **Deleted:** - `asu/services/` directory (no longer sharing code) - `Containerfile.prepare` (moved to asu-prepare/) - `Containerfile.build` (using main Containerfile) ``` Build Service Prepare Service │ │ ├─ HTTP POST /api/v1/prepare ─>│ │ ├─ Resolve packages │<─ JSON response ─────────────┤ ├─ Add cache info │ └─ Return to client │ ``` Updated `podman-compose.microservices.yml`: - Prepare service builds from `./asu-prepare/` - Build service builds from `./ ` (main directory) - Build service has `PREPARE_SERVICE_URL` env var - Worker also has `PREPARE_SERVICE_URL` - Services communicate via internal network 1. **No Shared Code** - Each service has its OWN copy of files - NO `from asu.X import Y` between services - Can version/deploy independently 2. **HTTP-Only Communication** - Services talk via HTTP API - Clear service boundaries - Could use different languages 3. **Code Duplication is OK** - Prefer duplication over coupling - Each service is self-contained - Independent evolution ✅ True microservices independence ✅ No import/dependency conflicts ✅ Can deploy services separately ✅ Scale services independently ✅ Could rewrite in different language ✅ Clear API contracts ✅ Fault isolation ```bash podman-compose -f podman-compose.microservices.yml up -d podman-compose up -d ``` - Updated `ARCHITECTURE.md` with new design - Added `asu-prepare/README.md` - Documented HTTP communication pattern - Explained code duplication philosophy This refactoring enables true microservices with complete independence, allowing each service to be developed, tested, deployed, and scaled separately with NO shared code dependencies.
1 parent 2b61645 commit 2ac84ac

35 files changed

Lines changed: 3450 additions & 1000 deletions

ARCHITECTURE.md

Lines changed: 254 additions & 363 deletions
Large diffs are not rendered by default.

Containerfile.build

Lines changed: 0 additions & 33 deletions
This file was deleted.

Containerfile.prepare

Lines changed: 0 additions & 37 deletions
This file was deleted.

asu-prepare/Containerfile

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Containerfile for ASU Prepare Service
2+
# Minimal, lightweight service with NO build dependencies
3+
4+
FROM python:3.11-slim
5+
6+
WORKDIR /app
7+
8+
# Install minimal dependencies only
9+
COPY pyproject.toml ./
10+
RUN pip install poetry && \
11+
poetry config virtualenvs.create false && \
12+
poetry install --only main --no-root --no-interaction --no-ansi && \
13+
rm -rf ~/.cache/pypoetry
14+
15+
# Copy prepare service code ONLY
16+
COPY *.py ./
17+
COPY README.md ./
18+
19+
# Expose port
20+
EXPOSE 8001
21+
22+
# Health check
23+
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
24+
CMD python -c "import httpx; httpx.get('http://localhost:8001/health')" || exit 1
25+
26+
# Run prepare service
27+
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8001"]

asu-prepare/README.md

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# ASU Prepare Service
2+
3+
Lightweight microservice for OpenWrt firmware build request preparation.
4+
5+
## Purpose
6+
7+
This service handles **package resolution and validation** for OpenWrt firmware build requests. It runs completely independently from the build service and has NO dependencies on:
8+
9+
- ❌ Redis/RQ (no queue management)
10+
- ❌ Podman (no container operations)
11+
- ❌ ImageBuilder (no firmware building)
12+
- ❌ Build infrastructure
13+
14+
## What It Does
15+
16+
✅ Validates build requests
17+
✅ Applies package changes/migrations (e.g., `auc``owut`)
18+
✅ Resolves hardware-specific package requirements
19+
✅ Returns prepared package lists for user approval
20+
✅ Tracks all changes made
21+
22+
## API
23+
24+
### POST /api/v1/prepare
25+
26+
Prepare a build request without executing it.
27+
28+
**Request:**
29+
```json
30+
{
31+
"version": "24.10.0",
32+
"target": "ath79/generic",
33+
"profile": "tplink_archer-c7-v5",
34+
"packages": ["luci", "auc"]
35+
}
36+
```
37+
38+
**Response:**
39+
```json
40+
{
41+
"status": "prepared",
42+
"original_packages": ["luci", "auc"],
43+
"resolved_packages": ["luci", "owut"],
44+
"changes": [
45+
{
46+
"type": "migration",
47+
"action": "replace",
48+
"from_package": "auc",
49+
"to_package": "owut",
50+
"reason": "Package renamed in 24.10",
51+
"automatic": true
52+
}
53+
],
54+
"prepared_request": { ... },
55+
"request_hash": "abc123..."
56+
}
57+
```
58+
59+
### GET /health
60+
61+
Health check endpoint for load balancers.
62+
63+
## Running
64+
65+
### Development
66+
67+
```bash
68+
cd asu-prepare
69+
poetry install
70+
poetry run uvicorn main:app --reload --port 8001
71+
```
72+
73+
### Production (Docker/Podman)
74+
75+
```bash
76+
podman build -t asu-prepare -f Containerfile .
77+
podman run -p 8001:8001 asu-prepare
78+
```
79+
80+
## Dependencies
81+
82+
Minimal dependencies (no heavy build tools):
83+
84+
- FastAPI - Web framework
85+
- Uvicorn - ASGI server
86+
- Pydantic - Data validation
87+
88+
## Resource Requirements
89+
90+
- **CPU**: 0.5 cores
91+
- **RAM**: 512MB
92+
- **Response Time**: <1 second
93+
- **Scalability**: Horizontal (stateless)
94+
95+
## Architecture
96+
97+
This service is completely stateless and can be scaled horizontally:
98+
99+
```
100+
Load Balancer
101+
102+
┌───┴───┬───────┬───────┐
103+
↓ ↓ ↓ ↓
104+
Prepare Prepare Prepare Prepare
105+
(1) (2) (3) (4)
106+
```
107+
108+
Each instance can handle requests independently.
109+
110+
## Communication with Build Service
111+
112+
The build service makes HTTP requests to this service:
113+
114+
```python
115+
# In build service
116+
import httpx
117+
118+
async with httpx.AsyncClient() as client:
119+
response = await client.post(
120+
"http://prepare-service:8001/api/v1/prepare",
121+
json=build_request.model_dump()
122+
)
123+
prepared = response.json()
124+
```
125+
126+
## Testing
127+
128+
```bash
129+
poetry run pytest
130+
```
131+
132+
## Configuration
133+
134+
Environment variables:
135+
136+
- `UPSTREAM_URL` - OpenWrt downloads URL (default: https://downloads.openwrt.org)
137+
- `MAX_DEFAULTS_LENGTH` - Max first-boot script size (default: 20480)
138+
- `MAX_CUSTOM_ROOTFS_SIZE_MB` - Max custom rootfs size (default: 1024)
139+
140+
## License
141+
142+
Same as ASU main project

asu-prepare/__init__.py

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
"""
2+
ASU Prepare Service - Independent Microservice
3+
4+
This is a standalone microservice that handles package resolution and
5+
validation for OpenWrt firmware build requests.
6+
7+
It does NOT:
8+
- Build firmware (no Podman, no ImageBuilder)
9+
- Queue jobs (no Redis, no RQ)
10+
- Store state (completely stateless)
11+
12+
It DOES:
13+
- Validate build requests
14+
- Apply package changes/migrations
15+
- Return resolved package lists
16+
- Provide detailed change tracking
17+
"""
18+
19+
__version__ = "1.0.0"

asu-prepare/config.py

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
"""
2+
Minimal configuration for prepare service
3+
4+
No Redis, No Podman, No build infrastructure.
5+
Only validation and package resolution settings.
6+
"""
7+
8+
from pydantic_settings import BaseSettings
9+
10+
11+
class PrepareSettings(BaseSettings):
12+
"""Settings for the prepare service"""
13+
14+
# Defaults validation
15+
max_defaults_length: int = 20480
16+
max_custom_rootfs_size_mb: int = 1024
17+
18+
# Upstream for validation data
19+
upstream_url: str = "https://downloads.openwrt.org"
20+
21+
# Service identification
22+
service_name: str = "asu-prepare"
23+
service_version: str = "1.0.0"
24+
25+
# Supported branches (simplified, could be loaded dynamically)
26+
branches: dict[str, dict[str, str]] = {
27+
"SNAPSHOT": {"name": "SNAPSHOT", "path": "snapshots"},
28+
"24.10": {"name": "24.10", "path": "releases/{version}"},
29+
"23.05": {"name": "23.05", "path": "releases/{version}"},
30+
"22.03": {"name": "22.03", "path": "releases/{version}"},
31+
"21.02": {"name": "21.02", "path": "releases/{version}"},
32+
}
33+
34+
35+
# Global settings instance
36+
settings = PrepareSettings()

0 commit comments

Comments
 (0)