Skip to content

feat(api): implement privacy-safe paginated analytics endpoint#19

Open
Samuel-21-eng wants to merge 3 commits into
emrekayat:mainfrom
Samuel-21-eng:feature/paginated-privacy-analystics
Open

feat(api): implement privacy-safe paginated analytics endpoint#19
Samuel-21-eng wants to merge 3 commits into
emrekayat:mainfrom
Samuel-21-eng:feature/paginated-privacy-analystics

Conversation

@Samuel-21-eng

Copy link
Copy Markdown

🚀 Overview
Implements a privacy-safe, cursor-paginated analytics engine across the workspace. This updates the backend architecture to securely stream usage logs from the file-based persistence layer (db.json) while masking sensitive network details before data reaches the client dashboard.

🛠️ Key Changes

  • packages/shared:
    • Exported type-safe definitions for PrivacySafeAnalyticsRecord and PaginatedAnalyticsResponse to enforce strict consistency between frontend layouts and backend data streams.
  • apps/api (Services):
    • Implemented a data-redaction filter in analytics-privacy.ts to securely mask Stellar wallet addresses down to their leading and trailing segments.
    • Developed a slice-window cursor strategy in query-service.ts to cleanly process variable page sizes (limit) and forward pagination references (cursor) against the native storage engine.
  • apps/api (Routes):
    • Replaced the placeholder /api/analytics summary handler in public.ts with an active, asynchronous controller linking query params to our pagination processor.

🧪 Verification

  • Checked the cross-workspace types engine via npm run typecheck — compilation successful with 0 errors.

closes #10

@vercel

vercel Bot commented Jun 27, 2026

Copy link
Copy Markdown

@VictorEzenma is attempting to deploy a commit to the emrekayat's projects Team on Vercel.

A member of the Team first needs to authorize it.

@drips-wave

drips-wave Bot commented Jun 27, 2026

Copy link
Copy Markdown

@Samuel-21-eng Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@emrekayat

Copy link
Copy Markdown
Owner

Thanks for opening this for #10. I reviewed the diff and cannot merge it yet.

Blocking issues:

  • packages/shared/src/types.ts removes existing sponsorship types and defines PrivacySafeAnalyticsRecord / PaginatedAnalyticsResponse twice, which would break existing consumers.
  • /api/analytics is changed from the existing aggregate summary to only a paginated list response, so demo/verified/settled/failed counts and on-chain volume separation are not implemented.
  • Public records still derive volumeType from mode === "demo", but mode is search/news/scrape in the current usage model. This will misclassify demo vs settled payments.
  • There are no tests for aggregation, redaction, pagination boundaries, retention, or unauthorized detail access.
  • Retention/documentation/dashboard updates from the acceptance criteria are missing.

Please rebase on latest main, preserve the existing shared sponsorship types, add the analytics types once, keep aggregate summary fields while adding cursor-paginated safe history, and cover the criteria with tests.

@emrekayat

Copy link
Copy Markdown
Owner

Rechecked this for #10 after the latest main updates. It still cannot be merged.

The branch is now conflicting with main, and the implementation is still based on the old db.json persistence path. Since #20 moved analytics/payment persistence to the SQLite-backed storage adapter, this needs a fresh rebase and an implementation on top of the new storage APIs.

The earlier blockers also still apply: preserve existing shared types, keep aggregate analytics while adding privacy-safe paginated history, classify demo/verified/settled/failed from payment status/evidence rather than query mode, and add tests/docs/dashboard updates for the #10 acceptance criteria.

@emrekayat

Copy link
Copy Markdown
Owner

Rechecked the latest head after the new main merge. I still cannot merge this PR for #10.

Command run:

  • npm test --workspace @query402/api -- src/routes/public.test.ts

Current blocker: the public route test suite fails at module load with ReferenceError: MAX_USAGE_EVENTS is not defined from apps/api/src/routes/public.ts. So the branch is not currently testable/mergeable.

I also noticed two code-quality issues that should be cleaned up while fixing this:

  • packages/shared/src/types.ts defines PrivacySafeAnalyticsRecord and PaginatedAnalyticsResponse twice.
  • package-lock.json has unrelated peer-flag churn; please keep lockfile changes limited to actual dependency changes.

Once the route loads cleanly, please add/keep tests for the issue #10 privacy requirements: no raw query text or scrape URLs, no full payer address, validated limit/cursor behavior, and demo vs settled separation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add privacy-safe, paginated analytics with demo/on-chain separation

3 participants