Merge pull request #100 from techdiary-dev/copilot/migrate-article-cleanup-cron

Migrate article cleanup cron from Cloudflare Worker to Inngest

Author: kingRayhan

CLAUDE.md

@@ -141,7 +141,11 @@ Cloudflare R2 (S3-compatible):
 - `S3_BUCKET` - R2 bucket name
 - `S3_ACCESS_KEY_ID` - R2 access key
 - `S3_SECRET_ACCESS_KEY` - R2 secret key
-- `CRON_SECRET` - Shared secret for `x-cron-secret` header on cron API endpoint
+
+Inngest:
+
+- `INNGEST_EVENT_KEY` - Inngest event key (optional for local dev)
+- `INNGEST_SIGNING_KEY` - Inngest signing key (optional for local dev)
 
 ## Key Features Implementation
 
@@ -361,8 +365,8 @@ const feedQuery = useInfiniteQuery({
 - **Bengali Language Support**: Custom font loading (Kohinoor Bangla) and i18n in `src/i18n/`
 - **SEO Optimization**: Dynamic sitemaps in `src/app/sitemaps/`, Open Graph tags, and schema markup
 - **No test framework**: There are no automated tests — use `bun run play` for backend experimentation
-- **Cloudflare Workers**: `wrangler.toml` configures a separate cron worker (`src/workers/cron-worker.ts`). `bun run wrangler:dev` starts it locally. The worker fires on `0 2 * * *` and calls `POST /api/cron/cleanup-articles` with `x-cron-secret` header.
-- **Article soft-delete**: Articles have a `delete_scheduled_at` field. Setting it schedules permanent deletion; `article-cleanup-service.ts` processes them when the cron fires. Use `restoreScheduleDeletedArticle` to cancel.
+- **Cloudflare Workers**: `wrangler.toml` still references `src/workers/cron-worker.ts` but the cron trigger has been removed. Article cleanup is now handled by Inngest (see below).
+- **Article soft-delete**: Articles have a `delete_scheduled_at` field. Setting it schedules permanent deletion; `article-cleanup-service.ts` processes them when the **Inngest cron** (`cleanup-expired-articles`, `0 2 * * *` UTC) fires. Use `restoreScheduleDeletedArticle` to cancel.
 
 ## Caching & ISR (Incremental Static Regeneration)
 

CLOUDFLARE_CRON_SETUP.md

@@ -1,143 +1,28 @@
-# Cloudflare Cron Setup for Article Cleanup
+# Article Cleanup Cron Setup
 
-This document explains how to set up Cloudflare Cron Triggers to automatically delete expired articles.
+> **Note**: The Cloudflare Worker cron for article cleanup has been retired.
+> Article cleanup is now handled by an **Inngest cron function** that runs daily at 2:00 AM UTC.
 
-## Overview
+## Current Setup
 
-The system consists of:
-1. **API Route**: `/api/cron/cleanup-articles` - Handles the actual cleanup logic
-2. **Cloudflare Worker**: `src/workers/cron-worker.ts` - Cron trigger that calls the API
-3. **Backend Service**: `src/backend/services/article-cleanup-service.ts` - Database operations
+Article cleanup runs as an Inngest cron function registered at `/api/inngest`.
 
-## Setup Instructions
+- **Function ID**: `cleanup-expired-articles`
+- **Schedule**: `0 2 * * *` (daily at 2:00 AM UTC)
+- **Source**: `src/lib/inngest.ts` → `cleanupExpiredArticlesFn`
+- **Logic**: `src/backend/services/article-cleanup-service.ts` → `deleteExpiredArticles()`
 
-### 1. Install Wrangler CLI
+## Manual Runs
 
-```bash
-npm install -g wrangler
-# or
-bun install -g wrangler
-```
-
-### 2. Authenticate with Cloudflare
-
-```bash
-wrangler login
-```
-
-### 3. Configure Environment Variables
-
-Update `wrangler.toml` with your actual domain:
-
-```toml
-[vars]
-CRON_TARGET_URL = "https://your-actual-domain.com/api/cron/cleanup-articles"
-```
-
-### 4. Set Secret (Optional but Recommended)
-
-For additional security, set a secret that the cron worker will send:
-
-```bash
-wrangler secret put CRON_SECRET
-# Enter your secret when prompted
-```
-
-Then add the same secret to your Next.js environment:
-
-```bash
-# .env.local
-CRON_SECRET=your-secret-key
-```
-
-### 5. Deploy the Cron Worker
-
-```bash
-wrangler deploy src/workers/cron-worker.ts
-```
-
-### 6. Verify Cron Schedule
-
-The cron is configured to run daily at 2:00 AM UTC. You can modify the schedule in `wrangler.toml`:
-
-```toml
-[triggers]
-crons = [
-  "0 2 * * *"  # Daily at 2:00 AM UTC
-]
-```
-
-Other common schedules:
-- `"0 * * * *"` - Every hour
-- `"0 2 * * 0"` - Every Sunday at 2:00 AM
-- `"0 2 1 * *"` - First day of every month at 2:00 AM
-
-### 7. Monitor Cron Executions
-
-You can monitor executions in the Cloudflare dashboard:
-1. Go to Workers & Pages
-2. Select your worker
-3. Check the "Logs" tab for execution logs
-
-## Manual Testing
-
-You can test the cleanup manually by calling the API directly:
-
-```bash
-# GET request for testing
-curl https://your-domain.com/api/cron/cleanup-articles
-
-# POST request (mimics cron call)
-curl -X POST https://your-domain.com/api/cron/cleanup-articles \
-  -H "Content-Type: application/json" \
-  -H "x-cron-secret: your-secret-key"
-```
+Trigger a manual run from the [Inngest dashboard](https://app.inngest.com) by invoking the `cleanup-expired-articles` function.
 
 ## How It Works
 
 1. **Article Scheduling**: Articles can be scheduled for deletion by setting the `delete_scheduled_at` field
-2. **Cron Trigger**: Cloudflare runs the worker daily at 2:00 AM UTC
-3. **API Call**: Worker calls `/api/cron/cleanup-articles` endpoint
-4. **Cleanup Logic**: Service finds articles where `delete_scheduled_at < current_time` and deletes them
-5. **Response**: API returns count of deleted articles and their details
-
-## Database Schema
-
-Articles are scheduled for deletion using the `delete_scheduled_at` timestamp field:
-
-```sql
--- Example of scheduling an article for deletion in 30 days
-UPDATE articles 
-SET delete_scheduled_at = NOW() + INTERVAL '30 days'
-WHERE id = 'article-id';
-```
-
-## Additional Functions
-
-The cleanup service also provides:
-- `scheduleArticleForDeletion(articleId, deleteAt)` - Schedule an article
-- `cancelScheduledDeletion(articleId)` - Cancel scheduled deletion  
-- `getScheduledArticles()` - Get all articles scheduled for deletion
-
-## Troubleshooting
-
-### Common Issues
-
-1. **404 Error**: Check that your `CRON_TARGET_URL` is correct
-2. **401 Unauthorized**: Verify `CRON_SECRET` matches between worker and API
-3. **500 Error**: Check database connection and permissions
-4. **No Articles Deleted**: Verify articles have `delete_scheduled_at` set and are past due
-
-### Debugging
-
-1. Check Cloudflare Worker logs in the dashboard
-2. Check your Next.js application logs
-3. Test the API endpoint manually
-4. Verify database records with `getScheduledArticles()`
+2. **Inngest Cron**: Inngest triggers the function daily at 2:00 AM UTC
+3. **Cleanup Logic**: `deleteExpiredArticles()` finds articles where `delete_scheduled_at < current_time` and deletes them from the database and MeiliSearch
 
-## Security Considerations
+## Required Environment Variables
 
-1. Use `CRON_SECRET` to verify requests come from your worker
-2. Consider rate limiting the endpoint
-3. Log all cleanup operations for audit trails
-4. Ensure proper database permissions for the cleanup operations
+- `INNGEST_EVENT_KEY` — Inngest event key (optional for local dev)
+- `INNGEST_SIGNING_KEY` — Inngest signing key (optional for local dev)

src/app/api/cron/cleanup-articles/route.ts

@@ -1,7 +1,11 @@
 import { serve } from "inngest/next";
-import { inngest, persistNotificationFn } from "@/lib/inngest";
+import {
+  inngest,
+  persistNotificationFn,
+  cleanupExpiredArticlesFn,
+} from "@/lib/inngest";
 
 export const { GET, POST, PUT } = serve({
   client: inngest,
-  functions: [persistNotificationFn],
+  functions: [persistNotificationFn, cleanupExpiredArticlesFn],
 });

src/app/api/inngest/route.ts

@@ -1,5 +1,5 @@
 "use server";
-import { and, lt, neq } from "sqlkit";
+import { and, lt, lte, neq } from "sqlkit";
 import { persistenceRepository } from "../persistence/persistence-repositories";
 import { handleActionException } from "./RepositoryException";
 import { deleteArticleById } from "./search.service";
@@ -14,7 +14,7 @@ export async function deleteExpiredArticles() {
     const articlesToDelete = await persistenceRepository.article.find({
       where: and(
         neq("delete_scheduled_at", null),
-        lt("delete_scheduled_at", currentTime)
+        lte("delete_scheduled_at", currentTime),
       ),
     });
 
@@ -25,12 +25,12 @@ export async function deleteExpiredArticles() {
     const deleteResult = await persistenceRepository.article.delete({
       where: and(
         neq("delete_scheduled_at", null),
-        lt("delete_scheduled_at", currentTime)
+        lte("delete_scheduled_at", currentTime),
       ),
     });
 
     console.log(
-      `Successfully deleted ${deleteResult?.rowCount} expired articles`
+      `Successfully deleted ${deleteResult?.rowCount} expired articles`,
     );
 
     return {

src/backend/services/article-cleanup-service.ts

@@ -71,7 +71,7 @@ export const syncArticleById = async (articleId: string) => {
 
     if (!article) {
       throw new Error(
-        `Article with ID ${articleId} not found or not published.`
+        `Article with ID ${articleId} not found or not published.`,
       );
     }
 
@@ -91,6 +91,7 @@ export const syncArticleById = async (articleId: string) => {
     console.error(`Error syncing article ${articleId}:`, error);
   }
 };
+
 export const deleteArticleById = async (articleId: string) => {
   try {
     const response = await index.deleteDocument(articleId);

src/backend/services/search.service.ts

@@ -20,8 +20,6 @@ export const env = createEnv({
     S3_ACCESS_SECRET: z.string().min(1),
     S3_BUCKET: z.string().min(1),
 
-    CRON_SECRET: z.string().optional(),
-
     // Inngest
     INNGEST_EVENT_KEY: z.string().optional(),
     INNGEST_SIGNING_KEY: z.string().optional(),
@@ -50,8 +48,6 @@ export const env = createEnv({
     S3_ACCESS_SECRET: process.env.S3_ACCESS_SECRET,
     S3_BUCKET: process.env.S3_BUCKET,
 
-    CRON_SECRET: process.env.CRON_SECRET,
-
     INNGEST_EVENT_KEY: process.env.INNGEST_EVENT_KEY,
     INNGEST_SIGNING_KEY: process.env.INNGEST_SIGNING_KEY,
   },

src/env.ts

@@ -7,6 +7,7 @@ import {
 import { persistenceRepository } from "@/backend/persistence/persistence-repositories";
 import { ActionException } from "@/backend/services/RepositoryException";
 import { buildPersistableNotification } from "@/backend/services/notifications.payload";
+import { deleteExpiredArticles } from "@/backend/services/article-cleanup-service";
 
 const notificationPayloadSchema = z.object({
   article_id: z.string().optional(),
@@ -101,6 +102,17 @@ export const inngest = new Inngest({
   eventKey: process.env.INNGEST_EVENT_KEY ?? "local",
 });
 
+export const cleanupExpiredArticlesFn = inngest.createFunction(
+  {
+    id: "cleanup-expired-articles",
+    triggers: [{ cron: "0 2 * * *" }],
+  },
+  async () => {
+    const result = await deleteExpiredArticles();
+    return { deletedCount: result.deletedCount };
+  },
+);
+
 export const persistNotificationFn = inngest.createFunction(
   {
     id: "persist-notification",

src/lib/inngest.ts

@@ -3,13 +3,5 @@ compatibility_date = "2024-01-01"
 main = "src/workers/cron-worker.ts"
 # R2 bucket configuration
 
-# Cron triggers configuration
-[triggers]
-crons = [
-  # Run article cleanup every day at 2:00 AM UTC
-  "0 2 * * *",
-  # "* * * * *", # Uncomment for production use
-]
-
 [observability.logs]
 enabled = true