Skip to content

Commit 07720f7

Browse files
pcholakovpcholakov
committed
Add TypeScript e2e OTEL tracing example
1 parent 23fb664 commit 07720f7

9 files changed

Lines changed: 529 additions & 0 deletions

File tree

‎.tools/run_node_tests.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ function npm_install_check() {
1010
}
1111

1212
npm_install_check $PROJECT_ROOT/typescript/basics
13+
npm_install_check $PROJECT_ROOT/typescript/tracing
1314

1415
npm_install_check $PROJECT_ROOT/typescript/templates/node
1516
npm_install_check $PROJECT_ROOT/typescript/templates/lambda

‎typescript/tracing/.gitignore‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Node
2+
node_modules
3+
dist
4+
5+
# debug
6+
npm-debug.log*
7+
8+
# env files
9+
.env*
10+
11+
# typescript
12+
*.tsbuildinfo
13+
14+
# Restate
15+
.restate
16+
restate-data

‎typescript/tracing/README.md‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# End-to-End OpenTelemetry Tracing with Restate
2+
3+
This example demonstrates distributed tracing across a fictional multi-tier system:
4+
5+
```
6+
┌──────────┐ ┌─────────────┐ ┌─────────────────┐ ┌────────────┐
7+
│ Client │────▶│ Restate │────▶│ Greeter Service │────▶│ Downstream │
8+
│ App │ │ Server │ │ (SDK/Node) │ │ Service │
9+
└──────────┘ └─────────────┘ └─────────────────┘ └────────────┘
10+
│ │ │ │
11+
│ │ │ │
12+
▼ ▼ ▼ ▼
13+
┌────────────────────────────────────────────────────────────────────────┐
14+
│ Jaeger │
15+
└────────────────────────────────────────────────────────────────────────┘
16+
```
17+
18+
**What gets traced:**
19+
20+
1. **Client App** - Creates the root span and injects W3C trace context into the Restate request
21+
2. **Restate Server** - Receives trace context, emits spans for ingress requests and handler invocations
22+
3. **Greeter Service** - SDK handler that creates custom spans and propagates context to downstream calls
23+
4. **Downstream Service** - Receives and logs the propagated trace headers
24+
25+
## Prerequisites
26+
27+
- Node.js 18+
28+
- Docker (for Jaeger)
29+
30+
## Setup
31+
32+
### 1. Start Jaeger
33+
34+
```bash
35+
docker run -d --name jaeger \
36+
-p 4317:4317 \
37+
-p 16686:16686 \
38+
jaegertracing/all-in-one:latest
39+
```
40+
41+
Jaeger UI will be available at `http://localhost:16686`
42+
43+
### 2. Install dependencies
44+
45+
```bash
46+
npm install
47+
```
48+
49+
### 3. Start Restate Server with tracing enabled
50+
51+
```bash
52+
npx @restatedev/restate-server --tracing-endpoint http://localhost:4317
53+
```
54+
55+
### 4. Start the downstream service (terminal 1)
56+
57+
```bash
58+
npm run downstream
59+
```
60+
61+
### 5. Start the Greeter service (terminal 2)
62+
63+
```bash
64+
npm run service
65+
```
66+
67+
### 6. Register the service with Restate
68+
69+
```bash
70+
npx @restatedev/restate deployments register http://localhost:9080
71+
```
72+
73+
### 7. Run the client
74+
75+
```bash
76+
npm run client Alice
77+
```
78+
79+
## Viewing Traces
80+
81+
After running the client, you'll see output like:
82+
83+
```
84+
Root Trace ID: abc123...
85+
View in Jaeger: `http://localhost:16686/trace/abc123...`
86+
```
87+
88+
Open the Jaeger link to see the complete distributed trace spanning all four components.
89+
90+
## What You'll See in Jaeger
91+
92+
The trace will show spans from all four services:
93+
94+
- **client-app**: The root `client-request` span
95+
- **Greeter**: Restate server spans for ingress, invoke, and journal operations
96+
- **restate-greeter-service**: Custom `Greeter.greet` span with events
97+
- **downstream-service**: `handle-request` span (may show errors due to 50% failure rate)
98+
99+
![Jaeger trace showing end-to-end spans](trace.png)
100+
101+
## Key Pattern: Extracting Trace Context in TypeScript SDK
102+
103+
The Restate server propagates W3C trace context to handlers via HTTP headers. In the TypeScript SDK, you need to manually extract this from `ctx.request().attemptHeaders`:
104+
105+
```typescript
106+
import { context, propagation, Context } from "@opentelemetry/api";
107+
108+
function extractTraceContext(ctx: restate.Context): Context {
109+
const headers = ctx.request().attemptHeaders;
110+
const traceparent = headers.get("traceparent");
111+
const tracestate = headers.get("tracestate");
112+
113+
const carrier: Record<string, string> = {};
114+
if (traceparent) {
115+
carrier["traceparent"] = Array.isArray(traceparent)
116+
? traceparent[0]
117+
: traceparent;
118+
}
119+
if (tracestate) {
120+
carrier["tracestate"] = Array.isArray(tracestate)
121+
? tracestate[0]
122+
: tracestate;
123+
}
124+
return propagation.extract(context.active(), carrier);
125+
}
126+
```
127+
128+
Then run your handler logic within that context:
129+
130+
```typescript
131+
const traceContext = extractTraceContext(ctx);
132+
return context.with(traceContext, () => {
133+
const span = tracer.startSpan("MyHandler");
134+
// ... your logic here, span is now a child of Restate's span
135+
});
136+
```
137+
138+
## Files
139+
140+
- `src/client.ts` - Client app that initiates traced requests
141+
- `src/restate-service.ts` - Restate Greeter service with OpenTelemetry instrumentation
142+
- `src/downstream.ts` - HTTP server with tracing and random failure rate

‎typescript/tracing/package.json‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{
2+
"name": "@restatedev/examples-tracing",
3+
"version": "0.0.1",
4+
"description": "End-to-end OpenTelemetry tracing with Restate",
5+
"license": "MIT",
6+
"author": "Restate developers",
7+
"email": "code@restate.dev",
8+
"type": "commonjs",
9+
"scripts": {
10+
"build": "tsc --noEmitOnError",
11+
"service": "tsx ./src/restate-service.ts",
12+
"client": "tsx ./src/client.ts",
13+
"downstream": "tsx ./src/downstream.ts"
14+
},
15+
"dependencies": {
16+
"@opentelemetry/api": "^1.9.0",
17+
"@opentelemetry/exporter-trace-otlp-grpc": "^0.57.0",
18+
"@opentelemetry/resources": "^1.30.0",
19+
"@opentelemetry/sdk-node": "^0.57.0",
20+
"@opentelemetry/semantic-conventions": "^1.28.0",
21+
"@restatedev/restate-sdk": "^1.10.2"
22+
},
23+
"devDependencies": {
24+
"@types/node": "^20.14.2",
25+
"tsx": "^4.19.2",
26+
"typescript": "^5.4.5"
27+
}
28+
}

‎typescript/tracing/src/client.ts‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
// OpenTelemetry must be initialized before other imports
2+
import { NodeSDK } from "@opentelemetry/sdk-node";
3+
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-grpc";
4+
import { Resource } from "@opentelemetry/resources";
5+
import { ATTR_SERVICE_NAME } from "@opentelemetry/semantic-conventions";
6+
import {
7+
trace,
8+
context,
9+
propagation,
10+
SpanKind,
11+
SpanStatusCode,
12+
} from "@opentelemetry/api";
13+
14+
const sdk = new NodeSDK({
15+
resource: new Resource({
16+
[ATTR_SERVICE_NAME]: "client-app",
17+
}),
18+
traceExporter: new OTLPTraceExporter({
19+
url: "http://localhost:4317",
20+
}),
21+
});
22+
23+
sdk.start();
24+
25+
const RESTATE_INGRESS = "http://localhost:8080";
26+
const tracer = trace.getTracer("client-app");
27+
28+
async function main() {
29+
const name = process.argv[2] || "World";
30+
31+
console.log("=== Client App ===");
32+
console.log(`Calling Restate Greeter service with name: ${name}`);
33+
34+
// Create the root span for this request
35+
const rootSpan = tracer.startSpan("client-request", {
36+
kind: SpanKind.CLIENT,
37+
attributes: {
38+
"request.name": name,
39+
},
40+
});
41+
42+
try {
43+
const result = await context.with(
44+
trace.setSpan(context.active(), rootSpan),
45+
async () => {
46+
const headers: Record<string, string> = {
47+
"Content-Type": "application/json",
48+
};
49+
50+
propagation.inject(context.active(), headers);
51+
console.log(`Injected W3C trace context headers:`, headers);
52+
53+
const traceId = rootSpan.spanContext().traceId;
54+
console.log(`Root Trace ID: ${traceId}`);
55+
console.log(`View in Jaeger: http://localhost:16686/trace/${traceId}`);
56+
console.log("");
57+
58+
const response = await fetch(`${RESTATE_INGRESS}/Greeter/greet`, {
59+
method: "POST",
60+
headers,
61+
body: JSON.stringify(name),
62+
});
63+
64+
if (!response.ok) {
65+
throw new Error(`HTTP ${response.status}: ${await response.text()}`);
66+
}
67+
68+
return response.json();
69+
},
70+
);
71+
72+
rootSpan.addEvent("response_received", {
73+
"response.value": JSON.stringify(result),
74+
});
75+
rootSpan.setStatus({ code: SpanStatusCode.OK });
76+
77+
console.log(`Response: ${JSON.stringify(result)}`);
78+
} catch (err) {
79+
rootSpan.setStatus({
80+
code: SpanStatusCode.ERROR,
81+
message: err instanceof Error ? err.message : "Unknown error",
82+
});
83+
console.error("Error:", err);
84+
process.exitCode = 1;
85+
} finally {
86+
rootSpan.end();
87+
await sdk.shutdown();
88+
}
89+
}
90+
91+
main();

‎typescript/tracing/src/downstream.ts‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
// OpenTelemetry must be initialized before other imports
2+
import { NodeSDK } from "@opentelemetry/sdk-node";
3+
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-grpc";
4+
import { Resource } from "@opentelemetry/resources";
5+
import { ATTR_SERVICE_NAME } from "@opentelemetry/semantic-conventions";
6+
import {
7+
trace,
8+
context,
9+
propagation,
10+
SpanKind,
11+
SpanStatusCode,
12+
} from "@opentelemetry/api";
13+
import { createServer } from "node:http";
14+
15+
const sdk = new NodeSDK({
16+
resource: new Resource({
17+
[ATTR_SERVICE_NAME]: "downstream-service",
18+
}),
19+
traceExporter: new OTLPTraceExporter({
20+
url: "http://localhost:4317",
21+
}),
22+
});
23+
24+
sdk.start();
25+
26+
const PORT = 3000;
27+
const FAILURE_RATE = 0.5; // 50% chance
28+
29+
const tracer = trace.getTracer("downstream-service");
30+
31+
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
32+
33+
const server = createServer((req, res) => {
34+
// Extract trace context from incoming headers
35+
const carrier: Record<string, string> = {};
36+
const traceparent = req.headers["traceparent"];
37+
const tracestate = req.headers["tracestate"];
38+
39+
if (traceparent) {
40+
carrier["traceparent"] = Array.isArray(traceparent)
41+
? traceparent[0]
42+
: traceparent;
43+
}
44+
if (tracestate) {
45+
carrier["tracestate"] = Array.isArray(tracestate)
46+
? tracestate[0]
47+
: tracestate;
48+
}
49+
50+
const traceContext = propagation.extract(context.active(), carrier);
51+
52+
// Run request handling within the extracted trace context
53+
context.with(traceContext, async () => {
54+
const span = tracer.startSpan("handle-request", {
55+
kind: SpanKind.SERVER,
56+
attributes: {
57+
"http.method": req.method,
58+
"http.url": req.url,
59+
},
60+
});
61+
62+
try {
63+
// Simulate some work
64+
await sleep(50 + Math.random() * 100);
65+
66+
// Random failure
67+
if (Math.random() < FAILURE_RATE) {
68+
span.setStatus({
69+
code: SpanStatusCode.ERROR,
70+
message: "Random failure",
71+
});
72+
span.addEvent("failure_triggered", { rate: FAILURE_RATE });
73+
74+
res.writeHead(500, { "Content-Type": "application/json" });
75+
res.end(
76+
JSON.stringify({
77+
error: "Random failure",
78+
receivedTrace: !!traceparent,
79+
}),
80+
);
81+
return;
82+
}
83+
84+
span.addEvent("processing_complete");
85+
span.setStatus({ code: SpanStatusCode.OK });
86+
87+
res.writeHead(200, { "Content-Type": "application/json" });
88+
res.end(JSON.stringify({ status: "ok", receivedTrace: !!traceparent }));
89+
} finally {
90+
span.end();
91+
}
92+
});
93+
});
94+
95+
server.listen(PORT, () => {
96+
console.log(`Downstream service listening on http://localhost:${PORT}`);
97+
console.log(`Failure rate: ${FAILURE_RATE * 100}%`);
98+
});
99+
100+
process.on("SIGTERM", () => {
101+
sdk.shutdown().then(() => process.exit(0));
102+
});

0 commit comments

Comments
 (0)