Skip to content

Commit cfa83db

Browse files
committed
feat: integrate ClickHouse analytics for resource tracking
- Added ClickHouse client dependency to package.json for resource analytics. - Updated resource analytics documentation to include new session type and ingestion methods. - Enhanced environment configuration to support ClickHouse connection parameters. - Implemented ResourceViewTracker component for tracking views on articles and gists. - Added analytics links in ArticleEditor and GistEditor for published resources.
1 parent 2aefe5a commit cfa83db

20 files changed

Lines changed: 1525 additions & 33 deletions

File tree

docs/resource-analytics-plan.md

Lines changed: 23 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -35,19 +35,25 @@ Single events table keyed by resource, not by article only.
3535

3636
```sql
3737
CREATE TABLE resource_views (
38-
resource_type LowCardinality(String), -- e.g. ARTICLE, GIST (allowlist)
39-
resource_id UUID,
40-
viewer_id Nullable(UUID), -- NULL when anonymous
41-
session_id String, -- stable anonymous bucket (cookie / hash)
42-
referrer Nullable(String),
43-
country_code Nullable(String), -- optional, from geo later
44-
viewed_at DateTime DEFAULT now()
38+
resource_type LowCardinality(String), -- e.g. ARTICLE, GIST (allowlist)
39+
resource_id UUID,
40+
session_type LowCardinality(String), -- ANON | AUTHENTICATED
41+
session_id String, -- ANON: client token; AUTHENTICATED: user id (UUID string)
42+
referrer Nullable(String),
43+
country_code Nullable(String),
44+
viewed_at DateTime DEFAULT now()
4545
)
4646
ENGINE = MergeTree()
4747
PARTITION BY toYYYYMM(viewed_at)
4848
ORDER BY (resource_type, resource_id, viewed_at);
4949
```
5050

51+
Existing tables with `viewer_id`: run these **one statement per HTTP request** (or `clickhouse-client`), in order:
52+
53+
1. `migrations/clickhouse-migrate-session-type-1-add-column.sql`
54+
2. `migrations/clickhouse-migrate-session-type-2-backfill.sql`
55+
3. `migrations/clickhouse-migrate-session-type-3-drop-viewer.sql`
56+
5157
**Deduped “views” metric (v1):** `uniq(session_id)` per calendar day per resource, for a chosen window:
5258

5359
```sql
@@ -68,10 +74,10 @@ Adjust interval via app-level parameter (7 / 30 / 90 / custom).
6874

6975
Append-on-each-load stores **one row per event**. That supports **two KPIs from the same table** without changing ingestion:
7076

71-
| Metric | Meaning | Per-day aggregation |
72-
|--------|---------|---------------------|
73-
| **Impressions** | Total times the page was loaded and sent an event (includes refreshes, repeat loads same session). | `count()` |
74-
| **Unique viewers** | Distinct sessions that saw the resource at least once that day. | `uniq(session_id)` |
77+
| Metric | Meaning | Per-day aggregation |
78+
| ------------------ | -------------------------------------------------------------------------------------------------- | ------------------- |
79+
| **Impressions** | Total times the page was loaded and sent an event (includes refreshes, repeat loads same session). | `count()` |
80+
| **Unique viewers** | Distinct sessions that saw the resource at least once that day. | `uniq(session_id)` |
7581

7682
**Single query — daily series for both** (same filter as above):
7783

@@ -99,7 +105,7 @@ ORDER BY date;
99105
### HTTP API
100106

101107
- **Route:** `POST /api/analytics/view` (or `pageview` if you prefer naming parity with #114).
102-
- **Body (JSON):** `{ "resource_type": "ARTICLE", "resource_id": "<uuid>", "session_id": "<string>" }`.
108+
- **Body (JSON):** `{ "resource_type": "ARTICLE", "resource_id": "<uuid>", "session_id": "<string>" }`. The server sets **`session_type`** and the canonical **`session_id`**: if the user is logged in, `session_type = AUTHENTICATED` and `session_id = user id`; otherwise `session_type = ANON` and `session_id` is the client token from the body.
103109
- **Behavior:** validate allowlist + UUID, insert one row, return **204** quickly. Optionally use **wait_end_of_query: false** (or fire-and-forget pattern) so the client never blocks on ClickHouse latency.
104110

105111
### Client
@@ -158,12 +164,14 @@ Keep **ownership checks** in one module (e.g. `assertResourceAnalyticsAccess`) s
158164

159165
Add to the server env schema (e.g. `@t3-oss/env-nextjs`):
160166

161-
- `CLICKHOUSE_HOST`
162-
- `CLICKHOUSE_DATABASE`
167+
- `CLICKHOUSE_HOST` (hostname only, no protocol)
168+
- `CLICKHOUSE_PORT` (optional, default `8123` in code)
169+
- `CLICKHOUSE_DATABASE` (optional, default `default`)
163170
- `CLICKHOUSE_USERNAME`
164171
- `CLICKHOUSE_PASSWORD`
172+
- `CLICKHOUSE_SECURE``true` | `false` for `https://` vs `http://`
165173

166-
Optional: `CLICKHOUSE_URL` if using a single connection string provider.
174+
DDL for the events table lives in `migrations/clickhouse-schema.sql` (run once against the instance).
167175

168176
When vars are missing, ingest route should **no-op or 503** consistently; dashboard should show a clear “analytics unavailable” state so local dev without ClickHouse still runs.
169177

migrations/clickhouse-schema.sql

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
-- Run once against your ClickHouse instance (e.g. clickhouse-client or HTTP POST to :8123).
2+
-- session_type: ANON = client token in session_id; AUTHENTICATED = user id (UUID string) in session_id.
3+
CREATE TABLE IF NOT EXISTS resource_views (
4+
resource_type LowCardinality(String),
5+
resource_id UUID,
6+
session_type LowCardinality(String),
7+
session_id String,
8+
referrer Nullable(String),
9+
country_code Nullable(String),
10+
viewed_at DateTime DEFAULT now()
11+
)
12+
ENGINE = MergeTree()
13+
PARTITION BY toYYYYMM(viewed_at)
14+
ORDER BY (resource_type, resource_id, viewed_at);

package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
"dependencies": {
1818
"@aws-sdk/client-s3": "^3.828.0",
1919
"@aws-sdk/s3-request-presigner": "^3.828.0",
20+
"@clickhouse/client": "^1.18.2",
2021
"@cloudinary/react": "^1.14.1",
2122
"@cloudinary/url-gen": "^1.21.0",
2223
"@codesandbox/sandpack-react": "^2.20.0",

src/app/[username]/[articleHandle]/page.tsx

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ import Link from "next/link";
2222
import { notFound } from "next/navigation";
2323
import type { Article, WithContext } from "schema-dts";
2424
import { eq } from "sqlkit";
25+
import { ResourceViewTracker } from "@/components/analytics/ResourceViewTracker";
2526
import ArticleSidebar from "./_components/ArticleSidebar";
2627
import EditArticleButton from "./_components/EditArticleButton";
2728
import {
@@ -130,6 +131,9 @@ const Page: NextPage<ArticlePageProps> = async ({ params }) => {
130131

131132
return (
132133
<>
134+
{article.published_at ? (
135+
<ResourceViewTracker resourceType="ARTICLE" resourceId={article.id} />
136+
) : null}
133137
<script
134138
type="application/ld+json"
135139
dangerouslySetInnerHTML={{ __html: JSON.stringify(jsonLd) }}
@@ -197,7 +201,9 @@ const Page: NextPage<ArticlePageProps> = async ({ params }) => {
197201
<ArticleDraftBylineLabel />
198202
)}
199203
<span className="mx-1.5">·</span>
200-
<ArticleReadingTime minutes={readingTime(article?.body ?? "")} />
204+
<ArticleReadingTime
205+
minutes={readingTime(article?.body ?? "")}
206+
/>
201207
</div>
202208
</div>
203209
</div>
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
import { getClickHouseClient, isClickHouseConfigured } from "@/backend/persistence/clickhouse.client";
2+
import { AnalyticsInput } from "@/backend/services/inputs/analytics.input";
3+
import { authID } from "@/backend/services/session.actions";
4+
import { NextResponse } from "next/server";
5+
6+
/**
7+
* Records one resource view event (append-only). No-ops with 204 when ClickHouse is not configured.
8+
*/
9+
export async function POST(req: Request) {
10+
if (!isClickHouseConfigured()) {
11+
return new NextResponse(null, { status: 204 });
12+
}
13+
14+
const client = getClickHouseClient();
15+
if (!client) {
16+
return new NextResponse(null, { status: 204 });
17+
}
18+
19+
let body: unknown;
20+
try {
21+
body = await req.json();
22+
} catch {
23+
return NextResponse.json({ error: "Invalid JSON" }, { status: 400 });
24+
}
25+
26+
const parsed = await AnalyticsInput.recordViewBody.safeParseAsync(body);
27+
if (!parsed.success) {
28+
return NextResponse.json(
29+
{ error: parsed.error.issues.map((i) => i.message).join(", ") },
30+
{ status: 400 },
31+
);
32+
}
33+
34+
const userId = await authID();
35+
const referrer = req.headers.get("referer");
36+
37+
const session =
38+
userId != null
39+
? { session_type: "AUTHENTICATED" as const, session_id: userId }
40+
: { session_type: "ANON" as const, session_id: parsed.data.session_id };
41+
42+
try {
43+
await client.insert({
44+
table: "resource_views",
45+
values: [
46+
{
47+
resource_type: parsed.data.resource_type,
48+
resource_id: parsed.data.resource_id,
49+
session_type: session.session_type,
50+
session_id: session.session_id,
51+
referrer: referrer ? referrer.slice(0, 2048) : null,
52+
country_code: null,
53+
},
54+
],
55+
format: "JSONEachRow",
56+
});
57+
} catch (e) {
58+
console.error("[analytics/view] ClickHouse insert failed:", e);
59+
return new NextResponse(null, { status: 500 });
60+
}
61+
62+
return new NextResponse(null, { status: 204 });
63+
}
Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
"use client";
2+
3+
import * as React from "react";
4+
import { getDashboardAnalyticsOverview } from "@/backend/services/analytics.actions";
5+
import {
6+
Card,
7+
CardContent,
8+
CardDescription,
9+
CardHeader,
10+
CardTitle,
11+
} from "@/components/ui/card";
12+
import {
13+
ChartContainer,
14+
ChartLegend,
15+
ChartLegendContent,
16+
ChartTooltip,
17+
ChartTooltipContent,
18+
type ChartConfig,
19+
} from "@/components/ui/chart";
20+
import { useTranslation } from "@/i18n/use-translation";
21+
import { actionPromisify } from "@/lib/utils";
22+
import { useQuery } from "@tanstack/react-query";
23+
import { CartesianGrid, Line, LineChart, XAxis } from "recharts";
24+
import {
25+
AnalyticsRangeControls,
26+
analyticsTimeRangeToQueryPayload,
27+
type AnalyticsTimeRange,
28+
} from "@/components/analytics/AnalyticsRangeControls";
29+
30+
const chartConfig = {
31+
unique_viewers: {
32+
label: "Unique viewers",
33+
color: "hsl(var(--chart-1))",
34+
},
35+
impressions: {
36+
label: "Impressions",
37+
color: "hsl(var(--chart-2))",
38+
},
39+
} satisfies ChartConfig;
40+
41+
export default function DashboardAnalyticsOverview() {
42+
const { _t } = useTranslation();
43+
const [timeRange, setTimeRange] = React.useState<AnalyticsTimeRange>({
44+
kind: "preset",
45+
days: 30,
46+
});
47+
48+
const rangePayload = analyticsTimeRangeToQueryPayload(timeRange);
49+
50+
const query = useQuery({
51+
queryKey: ["dashboard-analytics-overview", rangePayload],
52+
queryFn: () =>
53+
actionPromisify(getDashboardAnalyticsOverview({ ...rangePayload })),
54+
});
55+
56+
if (query.isPending) {
57+
return (
58+
<section id="dashboard-analytics" className="mb-10 space-y-4 scroll-mt-4">
59+
<div className="h-8 w-48 animate-pulse rounded-md bg-muted" />
60+
<div className="h-24 animate-pulse rounded-lg bg-muted" />
61+
<div className="h-[300px] animate-pulse rounded-lg bg-muted" />
62+
</section>
63+
);
64+
}
65+
66+
if (query.isError) {
67+
return (
68+
<section id="dashboard-analytics" className="mb-10 scroll-mt-4">
69+
<p className="text-sm text-destructive">
70+
{query.error instanceof Error ? query.error.message : String(query.error)}
71+
</p>
72+
</section>
73+
);
74+
}
75+
76+
const data = query.data;
77+
const chartData = data.series.map((row) => ({
78+
...row,
79+
dateLabel: row.date.slice(5),
80+
}));
81+
82+
const hasTrackedResources =
83+
data.publishedArticleCount > 0 || data.publicGistCount > 0;
84+
85+
return (
86+
<section id="dashboard-analytics" className="mb-10 scroll-mt-4">
87+
<div className="mb-4 flex flex-col gap-1 sm:flex-row sm:items-end sm:justify-between">
88+
<div>
89+
<h2 className="text-lg font-semibold tracking-tight">
90+
{_t("Reach & views")}
91+
</h2>
92+
<p className="text-sm text-muted-foreground">
93+
{_t("Across your published articles and public gists")} (
94+
{data.publishedArticleCount}{" "}
95+
{data.publishedArticleCount === 1 ? _t("article") : _t("articles")}
96+
{", "}
97+
{data.publicGistCount}{" "}
98+
{data.publicGistCount === 1 ? _t("gist") : _t("gists")}).{" "}
99+
<span className="text-foreground/80">
100+
{_t("Use Article analytics on each published row for a single post.")}
101+
</span>
102+
</p>
103+
</div>
104+
<AnalyticsRangeControls value={timeRange} onChange={setTimeRange} />
105+
</div>
106+
107+
{!data.clickhouseConfigured ? (
108+
<p className="mb-4 rounded-md border border-amber-500/40 bg-amber-500/10 px-3 py-2 text-sm text-amber-950 dark:text-amber-100">
109+
{_t(
110+
"View analytics are not configured (ClickHouse env missing). Totals below show reactions and bookmarks from the database only.",
111+
)}
112+
</p>
113+
) : null}
114+
115+
{data.clickhouseConfigured && !hasTrackedResources ? (
116+
<p className="mb-4 rounded-md border border-border bg-muted/40 px-3 py-2 text-sm text-muted-foreground">
117+
{_t(
118+
"Publish an article or create a public gist to start collecting view analytics.",
119+
)}
120+
</p>
121+
) : null}
122+
123+
<div className="grid gap-4 sm:grid-cols-2 lg:grid-cols-4">
124+
<Card className="rounded-lg border shadow-none">
125+
<CardHeader className="pb-2">
126+
<CardDescription>{_t("Unique viewers")}</CardDescription>
127+
<CardTitle className="text-2xl tabular-nums">
128+
{data.totals.unique_viewers}
129+
</CardTitle>
130+
</CardHeader>
131+
</Card>
132+
<Card className="rounded-lg border shadow-none">
133+
<CardHeader className="pb-2">
134+
<CardDescription>{_t("Impressions")}</CardDescription>
135+
<CardTitle className="text-2xl tabular-nums">
136+
{data.totals.impressions}
137+
</CardTitle>
138+
</CardHeader>
139+
</Card>
140+
<Card className="rounded-lg border shadow-none">
141+
<CardHeader className="pb-2">
142+
<CardDescription>{_t("Reactions")}</CardDescription>
143+
<CardTitle className="text-2xl tabular-nums">{data.reactions}</CardTitle>
144+
</CardHeader>
145+
</Card>
146+
<Card className="rounded-lg border shadow-none">
147+
<CardHeader className="pb-2">
148+
<CardDescription>{_t("Bookmarks")}</CardDescription>
149+
<CardTitle className="text-2xl tabular-nums">{data.bookmarks}</CardTitle>
150+
</CardHeader>
151+
</Card>
152+
</div>
153+
154+
<Card className="mt-4 rounded-lg border shadow-none">
155+
<CardHeader>
156+
<CardTitle className="text-base">{_t("Views over time")}</CardTitle>
157+
<CardDescription>
158+
{_t("Stacked across everything you track in this dashboard")}
159+
</CardDescription>
160+
</CardHeader>
161+
<CardContent>
162+
{!data.clickhouseConfigured || !hasTrackedResources ? (
163+
<p className="text-sm text-muted-foreground">
164+
{_t("Chart appears when view tracking is available for your content.")}
165+
</p>
166+
) : chartData.length === 0 ? (
167+
<p className="text-sm text-muted-foreground">
168+
{_t("No view data in this range yet.")}
169+
</p>
170+
) : (
171+
<ChartContainer config={chartConfig} className="h-[min(320px,50vh)] w-full">
172+
<LineChart
173+
accessibilityLayer
174+
data={chartData}
175+
margin={{ left: 8, right: 8, top: 8, bottom: 0 }}
176+
>
177+
<CartesianGrid vertical={false} />
178+
<XAxis
179+
dataKey="dateLabel"
180+
tickLine={false}
181+
axisLine={false}
182+
tickMargin={8}
183+
/>
184+
<ChartTooltip cursor={false} content={<ChartTooltipContent />} />
185+
<ChartLegend content={<ChartLegendContent />} />
186+
<Line
187+
type="monotone"
188+
dataKey="unique_viewers"
189+
stroke="var(--color-unique_viewers)"
190+
strokeWidth={2}
191+
dot={false}
192+
/>
193+
<Line
194+
type="monotone"
195+
dataKey="impressions"
196+
stroke="var(--color-impressions)"
197+
strokeWidth={2}
198+
dot={false}
199+
/>
200+
</LineChart>
201+
</ChartContainer>
202+
)}
203+
</CardContent>
204+
</Card>
205+
</section>
206+
);
207+
}

0 commit comments

Comments
 (0)