diff --git a/.agents/skills/docker-and-compose/SKILL.md b/.agents/skills/docker-and-compose/SKILL.md index 97c4488..16854ab 100644 --- a/.agents/skills/docker-and-compose/SKILL.md +++ b/.agents/skills/docker-and-compose/SKILL.md @@ -19,9 +19,26 @@ One multi-stage `Dockerfile` does everything the pipeline needs: | `publish` | Publishes the Presentation project | | `run` | The runtime image: non-root `$APP_UID`, `WORKDIR /app`, data in `/app/data`, a `HEALTHCHECK` on `/health/live` | -`docker-compose.yml` runs the app with `DOTNET_ENVIRONMENT=docker`, the `movie-reviews-data` volume a pinned OpenTelemetry Collector that receives the app's OTLP logs, traces and metrics, configured by `otel-collector.yaml`, and a pinned Aspire dashboard the collector forwards them to (`telemetry`). The dashboard UI is at , and the collector's OTLP ports are published on 4317 and 4318 so a `dotnet run` can send to it too. The dashboard runs with `ASPIRE_DASHBOARD_UNSECURED_ALLOW_ANONYMOUS` and the collector's OTLP ports take anything sent to them, on purpose: the compose file is for a developer's machine, with nothing to sign in to. Keep it that way rather than adding sign-in, and don't offer the compose file for a shared or deployed environment. Images are built by CI but not pushed anywhere; the template ships the Dockerfile so users can. +`docker-compose.yml` runs the app with `DOTNET_ENVIRONMENT=docker`, the `movie-reviews-data` volume a pinned OpenTelemetry Collector that receives the app's OTLP logs, traces and metrics, configured by `otel-collector.yaml`, and a pinned Aspire dashboard the collector forwards them to (`telemetry`). The dashboard UI and the collector's OTLP ports are published on the template's own ports (below), so a `dotnet run` can send to the collector too. The dashboard runs with `ASPIRE_DASHBOARD_UNSECURED_ALLOW_ANONYMOUS` and the collector's OTLP ports take anything sent to them, on purpose: the compose file is for a developer's machine, with nothing to sign in to. Keep it that way rather than adding sign-in, and don't offer the compose file for a shared or deployed environment. Images are built by CI but not pushed anywhere; the template ships the Dockerfile so users can. -The Lambda template's `run` stage is the AWS Lambda .NET base image instead. It runs as user `65534` from `/var/task`, keeps its database in `/tmp`, and has no `HEALTHCHECK`. Its compose file runs the collector and the Aspire dashboard as well. Compose maps the runtime interface emulator to port 9000, so the function can be invoked with a `POST` to `/2015-03-31/functions/function/invocations`. +The Lambda template's `run` stage is the AWS Lambda .NET base image instead. It runs as user `65534` from `/var/task`, keeps its database in `/tmp`, and has no `HEALTHCHECK`. Its compose file runs the collector and the Aspire dashboard as well. Compose maps the runtime interface emulator to 7350, so the function can be invoked with a `POST` to `/2015-03-31/functions/function/invocations`. + +### Ports + +Every port published on the developer's machine, by the compose file or by `launchSettings.json`, sits in the template's own block of ten, so any two Lean Mean Clean solutions run side by side. The blocks follow the template order: Minimal API 7310, MCP 7320, GraphQL 7330, Web API 7340, Lambda Function 7350. Each port has the same place in every block: + +| Offset | Port | +| --- | --- | +| +0 | The app over HTTP, from Docker Compose or `dotnet run` | +| +1 | The app over HTTPS, `dotnet run` only | +| +2 | The Aspire dashboard | +| +3 | The collector's OTLP over gRPC | +| +4 | The collector's OTLP over HTTP | +| +5 | The MCP Inspector | +| +6 | The MCP chat demo | +| +7 to +9 | Spare | + +`docs/RUN.md` lists the ports this solution uses. The app listens on its own port everywhere: under `dotnet run`, in the image (the `Dockerfile` sets `ASPNETCORE_HTTP_PORTS`, `EXPOSE` and the `HEALTHCHECK` to it) and in the compose file, so `docker run -p` maps it to the same number. Change all four together. The Lambda template is the one exception: Lambda's runtime interface emulator always listens on 8080 inside its container, so only the host side of its mapping is ours. The dashboard, the collector and other third-party images keep their usual ports inside (18888, 4317 and 4318), and only the host side of their mappings moves. A new service takes a spare port from the block, never a product's well-known one such as 3000, 5432 or 9000. `docker compose port ` prints where a port ended up. ## Steps @@ -42,7 +59,7 @@ The Lambda template's `run` stage is the AWS Lambda .NET base image instead. It ``` bash docker build . --target run --tag local-check -docker compose up -d --build && curl -fsS http://localhost:8080/health/ready -# Lambda: curl -fsS -X POST http://localhost:9000/2015-03-31/functions/function/invocations -d '{}' +docker compose up -d --build --wait # waits for the image's HEALTHCHECK +# Lambda: curl -fsS -X POST "http://$(docker compose port function 8080)/2015-03-31/functions/function/invocations" -d '{}' docker compose down ``` diff --git a/.agents/skills/lambda-emulator/SKILL.md b/.agents/skills/lambda-emulator/SKILL.md index 86c317f..5f11b34 100644 --- a/.agents/skills/lambda-emulator/SKILL.md +++ b/.agents/skills/lambda-emulator/SKILL.md @@ -9,7 +9,7 @@ metadata: ## The rule -The runtime image is built on the AWS Lambda .NET base image, which includes the runtime interface emulator. Running the image locally gives an endpoint that invokes the function exactly as Lambda would. `docker-compose.yml` maps it to port 9000 and turns sample data on. The image runs `GetMovies` unless you name another handler. Each function's handler string names the class Lambda Annotations generates for it, and `src/Presentation/serverless.template` lists them all. +The runtime image is built on the AWS Lambda .NET base image, which includes the runtime interface emulator. Running the image locally gives an endpoint that invokes the function exactly as Lambda would. `docker-compose.yml` maps it to port 7350 and turns sample data on. The image runs `GetMovies` unless you name another handler. Each function's handler string names the class Lambda Annotations generates for it, and `src/Presentation/serverless.template` lists them all. ## Steps @@ -17,7 +17,7 @@ The runtime image is built on the AWS Lambda .NET base image, which includes the 2. Invoke it by posting the request JSON to the emulator: ```bash - curl -X POST http://localhost:9000/2015-03-31/functions/function/invocations \ + curl -X POST http://localhost:7350/2015-03-31/functions/function/invocations \ -d '{ "query": "global", "offset": 0, "limit": 5 }' ``` @@ -25,14 +25,14 @@ The runtime image is built on the AWS Lambda .NET base image, which includes the ```bash docker build . --target run --tag cleanlambdafunction:local - docker run --rm -p 9000:8080 -e Database__SeedSampleData=true cleanlambdafunction:local \ + docker run --rm -p 7350:8080 -e Database__SeedSampleData=true cleanlambdafunction:local \ "Presentation::CleanLambdaFunction.Presentation.LambdaFunction.Handler_GetAuthors_Generated::GetAuthors" ``` 4. An HTTP API function (`HttpGetMovies`, `HttpGetAuthors`, `HttpGetReviews`) takes the API Gateway HTTP API payload 2.0 event, not the request record. Run it by its handler string as in step 3 and post the event API Gateway would send. The query string values are strings, as they are on the wire: ```bash - curl -X POST http://localhost:9000/2015-03-31/functions/function/invocations \ + curl -X POST http://localhost:7350/2015-03-31/functions/function/invocations \ -d '{ "version": "2.0", "routeKey": "GET /movies", "rawPath": "/movies", "queryStringParameters": { "query": "global", "limit": "5" }, "requestContext": { "http": { "method": "GET", "path": "/movies" } } }' ``` @@ -40,19 +40,19 @@ The runtime image is built on the AWS Lambda .NET base image, which includes the 5. Send a request that fails validation, such as `{ "limit": 0 }`, and check the error's `errorType` and `errorMessage`. An HTTP API function answers a `400` with `content-type: application/problem+json` instead, and `"limit": "0"` in the query string shows it. 6. Read the container's output with `docker compose logs function`. The runtime's start, invoke and error lines sit between our compact JSON events, one per line, at `Debug` because compose sets `Serilog__MinimumLevel__Default` (`serilog`). - Each invocation's trace, metrics and logs also land in the Aspire dashboard at , through the collector compose starts beside the function (`telemetry`). + Each invocation's trace, metrics and logs also land in the Aspire dashboard at , through the collector compose starts beside the function (`telemetry`). 7. Stop it with `docker compose down`. ## Don't - Use the emulator in place of tests. Anything you check by hand gets a test (`presentation-integration-tests`). -- Expose port 9000 beyond your machine. The emulator has no authentication. +- Expose port 7350 beyond your machine. The emulator has no authentication. - Deploy to try a change. The emulator runs the same image Lambda would. ## Check ``` bash docker compose up -d --build -curl -fsS -X POST http://localhost:9000/2015-03-31/functions/function/invocations -d '{}' +curl -fsS -X POST http://localhost:7350/2015-03-31/functions/function/invocations -d '{}' docker compose down ``` diff --git a/.agents/skills/serilog/SKILL.md b/.agents/skills/serilog/SKILL.md index d485899..857d8ed 100644 --- a/.agents/skills/serilog/SKILL.md +++ b/.agents/skills/serilog/SKILL.md @@ -14,7 +14,7 @@ Serilog is the logger. Application and Infrastructure code log through `Microsof | Environment | Sink | Where to look | | --- | --- | --- | | `local` | Console, with the output template in `appsettings.local.json` | The terminal | -| `docker` | Console, and OTLP through the collector to the Aspire dashboard | , or `docker compose logs` | +| `docker` | Console, and OTLP through the collector to the Aspire dashboard | The Aspire dashboard, on the port `docs/RUN.md` lists, or `docker compose logs` | | Lambda, anywhere | Console, as compact JSON, and OTLP when the endpoint is set | CloudWatch Logs, or `docker compose logs` for the emulator, and the Aspire dashboard in Docker Compose | | Any, when `OTEL_EXPORTER_OTLP_ENDPOINT` is set | Also OTLP, through `Serilog.Sinks.OpenTelemetry` | Whatever receives the OTLP (`telemetry`) | @@ -108,6 +108,6 @@ _ = builder.Services.AddSerilog((services, loggerConfiguration) => ``` bash dotnet run --project ./src/Presentation # watch the console, then make a request -docker compose up -d --build # then open http://localhost:18888 +docker compose up -d --build # then open the Aspire dashboard on the port docs/RUN.md lists dotnet test tests/Application.Tests.Unit # LoggingBehaviourTests check the messages with FakeLogger ``` diff --git a/.agents/skills/telemetry/SKILL.md b/.agents/skills/telemetry/SKILL.md index 34d0eb8..1c7e7ae 100644 --- a/.agents/skills/telemetry/SKILL.md +++ b/.agents/skills/telemetry/SKILL.md @@ -40,7 +40,7 @@ The Lambda template builds its tracer and meter providers once in `Startup` with The app sends to an OpenTelemetry Collector, never straight to a backend, because that's how it should run in production: the collector owns batching, retries, sampling, scrubbing and the choice of backend, so none of that lives in the app. `otel-collector.yaml` holds its configuration: an OTLP receiver on 4317 (gRPC) and 4318 (HTTP), the `memory_limiter` and `batch` processors, and one exporter per backend. -Locally, Docker Compose runs the collector (`otel/opentelemetry-collector`) and the Aspire dashboard (`mcr.microsoft.com/dotnet/aspire-dashboard`), and the collector forwards everything to the dashboard. The dashboard is only a container that receives OTLP and shows logs, traces and metrics at ; it keeps everything in memory, so it's empty after a restart. In production, run a collector beside the app or as a gateway and swap the dashboard exporter for your backend's. +Locally, Docker Compose runs the collector (`otel/opentelemetry-collector`) and the Aspire dashboard (`mcr.microsoft.com/dotnet/aspire-dashboard`), and the collector forwards everything to the dashboard. The dashboard is only a container that receives OTLP and shows logs, traces and metrics on the template's dashboard port (`docker-and-compose`); it keeps everything in memory, so it's empty after a restart. In production, run a collector beside the app or as a gateway and swap the dashboard exporter for your backend's. ## Steps @@ -51,7 +51,7 @@ Locally, Docker Compose runs the collector (`otel/opentelemetry-collector`) and 5. Use stable packages only. When the only instrumentation is a prerelease package, go without that signal and add it to the watch list below. 6. Test spans with an `ActivityListener` that keeps only the test's own trace (`unit-tests`). The integration tests send a `traceparent` header and check the request and use case spans join up. 7. To send telemetry somewhere new, add an exporter to `otel-collector.yaml` and list it in the pipelines. The app's settings don't change. -8. To see it locally, run `docker compose up -d --build` and open . For a `dotnet run`, start the collector with `docker compose up -d otel-collector` and set `OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317`. +8. To see it locally, run `docker compose up -d --build` and open the dashboard on the port `docs/RUN.md` lists. For a `dotnet run`, start the collector with `docker compose up -d otel-collector` and set `OTEL_EXPORTER_OTLP_ENDPOINT` to its gRPC port, `http://$(docker compose port otel-collector 4317)`. ## Example @@ -110,5 +110,5 @@ These signals and packages aren't stable for .NET yet. Check them when dependenc ``` bash dotnet test --filter "FullyQualifiedName~Telemetry|FullyQualifiedName~LoggingBehaviour" # use case span and end-to-end trace tests -docker compose up -d --build # then open http://localhost:18888 and make a request +docker compose up -d --build # then open the dashboard on the port docs/RUN.md lists and make a request ``` diff --git a/.github/workflows/build-pipeline.yml b/.github/workflows/build-pipeline.yml index e5c572f..6781e2e 100644 --- a/.github/workflows/build-pipeline.yml +++ b/.github/workflows/build-pipeline.yml @@ -355,9 +355,9 @@ jobs: # Start the image the way the README does and ask it for a page of movies. Scanning an # image proves nothing about whether it runs, so this is the check that it actually does. echo "smoke=failed" >> "$GITHUB_OUTPUT" - docker run --detach --name function --publish 9000:8080 --env Database__SeedSampleData=true "$IMAGE" + docker run --detach --name function --publish 7350:8080 --env Database__SeedSampleData=true "$IMAGE" for attempt in $(seq 1 30); do - if RESPONSE=$(curl --silent --fail --max-time 30 -X POST http://localhost:9000/2015-03-31/functions/function/invocations -d '{ "limit": 2 }'); then + if RESPONSE=$(curl --silent --fail --max-time 30 -X POST http://localhost:7350/2015-03-31/functions/function/invocations -d '{ "limit": 2 }'); then break fi sleep 1 diff --git a/AGENTS.md b/AGENTS.md index 18c69d2..a5b1e76 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,7 +27,7 @@ Run these before you say a change is done. They're what CI runs. dotnet build # warnings are errors dotnet test # every test project dotnet format --verify-no-changes # style, whitespace and usings -docker compose up -d --build # the function at http://localhost:9000, see lambda-emulator +docker compose up -d --build # the function at http://localhost:7350, see lambda-emulator ``` The `coverage` skill has the command that checks the 95% line coverage gate locally. diff --git a/NUGET.md b/NUGET.md index 31c383f..1ea4fc0 100644 --- a/NUGET.md +++ b/NUGET.md @@ -44,7 +44,7 @@ The image ships with the Lambda runtime emulator, so it answers exactly like Lam docker compose up -d --build # Ask it for a page of movies -curl -X POST http://localhost:9000/2015-03-31/functions/function/invocations \ +curl -X POST http://localhost:7350/2015-03-31/functions/function/invocations \ -d '{ "query": "space", "limit": 2 }' # Run the tests while you're here diff --git a/README.md b/README.md index bea5023..474fba1 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ The image ships with the Lambda runtime emulator, so it answers exactly like Lam docker compose up -d --build # Ask it for a page of movies -curl -X POST http://localhost:9000/2015-03-31/functions/function/invocations \ +curl -X POST http://localhost:7350/2015-03-31/functions/function/invocations \ -d '{ "query": "space", "limit": 2 }' ``` @@ -72,23 +72,23 @@ curl -X POST http://localhost:9000/2015-03-31/functions/function/invocations \ The sample movies are all made up, and the critics are harsh. -Every function returns one page at a time, with a `totalCount` and a `nextOffset` when there's more to come. Send the `nextOffset` back as `offset` to get the next page. Movies and authors come with their review count and average stars, worked out by the database rather than in a loop somewhere. Each invocation's logs, trace and metrics land in the [Aspire dashboard](https://learn.microsoft.com/dotnet/aspire/fundamentals/dashboard/standalone) at . +Every function returns one page at a time, with a `totalCount` and a `nextOffset` when there's more to come. Send the `nextOffset` back as `offset` to get the next page. Movies and authors come with their review count and average stars, worked out by the database rather than in a loop somewhere. Each invocation's logs, trace and metrics land in the [Aspire dashboard](https://learn.microsoft.com/dotnet/aspire/fundamentals/dashboard/standalone) at . The image runs `GetMovies` by default. The same image also has `GetAuthors` and `GetReviews`, and you pick one by passing its handler string when the container starts. [Lambda Annotations](https://github.com/aws/aws-lambda-dotnet/tree/master/Libraries/src/Amazon.Lambda.Annotations) writes the class behind each one, so you only write the method. ```bash -# Free up port 9000 and build the image +# Free up port 7350 and build the image docker compose down docker build . --target run --tag acme-reviews:local # Start the reviews function instead of movies -docker run --rm -p 9000:8080 \ +docker run --rm -p 7350:8080 \ -e Database__SeedSampleData=true \ acme-reviews:local \ "Presentation::WorksOnMyMachine.Reviews.Presentation.LambdaFunction.Handler_GetReviews_Generated::GetReviews" # Reviews take a movie or an author instead of a search term -curl -X POST http://localhost:9000/2015-03-31/functions/function/invocations \ +curl -X POST http://localhost:7350/2015-03-31/functions/function/invocations \ -d '{ "movieId": "cb188ef6-14e7-f85f-2f78-924c2ab26b28", "limit": 5 }' ``` diff --git a/docker-compose.yml b/docker-compose.yml index e745d04..d9f89b3 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -7,7 +7,7 @@ services: VERSION_PREFIX: 1.0.0.0 VERSION_SUFFIX: docker ports: - - "9000:8080" + - "7350:8080" environment: Database__SeedSampleData: "true" Serilog__MinimumLevel__Default: "Debug" @@ -20,13 +20,13 @@ services: volumes: - ./otel-collector.yaml:/etc/otelcol/config.yaml:ro ports: - - "4317:4317" - - "4318:4318" + - "7353:4317" + - "7354:4318" depends_on: - aspire-dashboard aspire-dashboard: image: mcr.microsoft.com/dotnet/aspire-dashboard:13.6.2 ports: - - "18888:18888" + - "7352:18888" environment: ASPIRE_DASHBOARD_UNSECURED_ALLOW_ANONYMOUS: "true" diff --git a/docs/RUN.md b/docs/RUN.md index 4d3ae66..158dcbd 100644 --- a/docs/RUN.md +++ b/docs/RUN.md @@ -25,7 +25,7 @@ Build the image and start it. ```bash docker build . --target run --tag cleanlambdafunction:local -docker run --rm -p 9000:8080 \ +docker run --rm -p 7350:8080 \ -e Database__SeedSampleData=true \ cleanlambdafunction:local ``` @@ -33,19 +33,19 @@ docker run --rm -p 9000:8080 \ Invoke the function from another terminal. ```bash -curl -X POST http://localhost:9000/2015-03-31/functions/function/invocations \ +curl -X POST http://localhost:7350/2015-03-31/functions/function/invocations \ -d '{ "query": "global", "offset": 0, "limit": 5 }' ``` To try one of the HTTP API functions, run the image with its handler and post an API Gateway HTTP API (payload 2.0) event. The emulator doesn't pretend to be API Gateway, so you send the event it would have sent. ```bash -docker run --rm -p 9000:8080 \ +docker run --rm -p 7350:8080 \ -e Database__SeedSampleData=true \ cleanlambdafunction:local \ "Presentation::CleanLambdaFunction.Presentation.LambdaFunction.Handler_HttpGetMovies_Generated::HttpGetMovies" -curl -X POST http://localhost:9000/2015-03-31/functions/function/invocations \ +curl -X POST http://localhost:7350/2015-03-31/functions/function/invocations \ -d '{ "version": "2.0", "routeKey": "GET /movies", "rawPath": "/movies", "queryStringParameters": { "query": "global", "limit": "5" }, "requestContext": { "http": { "method": "GET", "path": "/movies" } } }' ``` @@ -65,13 +65,26 @@ To shut it down: docker compose down ``` +## Ports + +Every Lean Mean Clean template has its own block of ten ports on your machine, so this solution runs beside one made from another template without a clash. This one uses 7350 to 7359: + +| Port | What listens | +| --- | --- | +| 7350 | The function, through the runtime interface emulator | +| 7352 | The Aspire dashboard | +| 7353 | The OpenTelemetry Collector, OTLP over gRPC | +| 7354 | The OpenTelemetry Collector, OTLP over HTTP | + +Inside its container the emulator still listens on 8080, as Lambda's base image expects, and the dashboard and collector keep their usual ports. To move a port, change the left-hand side of its mapping in `docker-compose.yml`. 7355 to 7359 are spare for anything you add. + ## Telemetry The function always collects [OpenTelemetry](https://opentelemetry.io) traces and metrics, and exports them, along with its logs, only when `OTEL_EXPORTER_OTLP_ENDPOINT` is set. Every other standard `OTEL_*` variable works too, such as `OTEL_SERVICE_NAME`, `OTEL_EXPORTER_OTLP_PROTOCOL` and `OTEL_EXPORTER_OTLP_HEADERS`. Logs always go to the console as compact JSON as well, so CloudWatch Logs keeps them whatever else you set. Each invocation is one trace: a span for the function from the [OpenTelemetry Lambda wrapper](https://github.com/open-telemetry/opentelemetry-dotnet-contrib/tree/main/src/OpenTelemetry.Instrumentation.AWSLambda), and one for the use case. Traces and metrics are flushed before the function returns, because Lambda can freeze the environment the moment it does. -`docker-compose.yml` also starts an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) and the [Aspire dashboard](https://learn.microsoft.com/dotnet/aspire/fundamentals/dashboard/standalone). The function sends everything to the collector, and the collector forwards it to the dashboard at . The dashboard is only a container that receives OTLP; nothing in the solution depends on Aspire. The dashboard has no login and the collector accepts OTLP from anyone on ports 4317 and 4318. That's on purpose for a developer's machine, so don't run this compose file anywhere other people can reach. `otel-collector.yaml` is the collector's configuration: an OTLP receiver, the memory limiter and batch processors, and one exporter to swap for your backend. +`docker-compose.yml` also starts an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) and the [Aspire dashboard](https://learn.microsoft.com/dotnet/aspire/fundamentals/dashboard/standalone). The function sends everything to the collector, and the collector forwards it to the dashboard at . The dashboard is only a container that receives OTLP; nothing in the solution depends on Aspire. The dashboard has no login and the collector accepts OTLP from anyone on ports 7353 and 7354. That's on purpose for a developer's machine, so don't run this compose file anywhere other people can reach. `otel-collector.yaml` is the collector's configuration: an OTLP receiver, the memory limiter and batch processors, and one exporter to swap for your backend. In AWS, send the function's telemetry to a collector rather than straight to a backend. Lambda container images can't use layers, so either build the [AWS Distro for OpenTelemetry](https://aws-otel.github.io/docs/getting-started/lambda) collector extension into the image, or run a collector as a gateway and point `OTEL_EXPORTER_OTLP_ENDPOINT` at it. Either way, the collector owns batching, retries and the choice of backend, X-Ray included, and changing any of that never touches the function.