Repository navigation
Conversation
…expose the actionable cap fetchExecutionsByClass logged a listing error and skipped the class, so a failed actionable query hid every pending approval while the response kept truncated=false. The class fetch now reports failure and fetchExecutionsAsHistory raises the incomplete-results signal (including on the no-executions early return). HistoryResponse gains actionable_executions_limit (config.MaxListLimit) next to executions_limit (terminal cap, DefaultListLimit), so the UI banner can quote the cap that was actually hit instead of always quoting the terminal cap. The banner text itself lives in frontend/ and is left to the UI owner. Refs LeanerCloud#686 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_cursor) GET /api/history listed at most `limit` purchase_history rows and 100 terminal executions, and rows past those caps could not be reached: there was no way to ask for the next page. Completed purchases and terminal executions are now one stream, newest first, ordered by (timestamp DESC, source DESC, row id DESC), which is unique. Each response lists `limit` rows of it and, when more rows the caller may see follow, sets has_more and next_cursor; passing next_cursor back as `cursor` returns the next page. The cursor is a keyset position, not an offset: the account scope and the execution filters run in Go after the SQL read, so an offset would count rows the caller never sees, and every row written ahead of it would shift the page. With the keyset, a later page resumes strictly after the last row shown, so rows written between page loads never repeat or skip a row, and a page costs the same at any depth. `offset` is rejected with 400 instead of ignored, so a client written against an offset API cannot loop on the first page. Actionable executions (pending approvals, in-flight purchases) are not paged: the first page lists the whole class as before, and cursor pages leave it out, so paging can never push a pending approval off the first page. The LeanerCloud#686 signals are kept: a failed execution listing or a full actionable class still sets truncated. has_more is judged on rows the caller may see, so other tenants' rows never set it, and the cursor only ever names a row the caller was shown. The scope is applied again on every page. Store: ListPurchaseHistoryPage (GetPurchaseHistoryFiltered's predicate plus the keyset, returning each row's id) and ListHistoryExecutionsPage (the terminal statuses, clean completed rows excluded, scheduled_date bounds pushed into SQL). No migration: the existing timestamp and scheduled_date indexes serve both. Closes LeanerCloud#674 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IAmJSD
added
priority/p2
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on fix/issue-686 (#1276). Until that merges, this diff also shows its commit (0ad90b7). Review only the last commit.
Root cause
GET /api/historyread at mostlimitpurchase_history rows (default 100, max 1000) and at most 100 terminal executions (failed, expired, canceled, audit-gap and partially completed). It returnedtruncated(#630) when a cap was hit, but it had no way to ask for the next rows, so older rows could not be reached at all. #632's scope note moved paging to this issue.Fix
One paged stream. Completed purchases and terminal executions are merged into one list, newest first. The order is
(timestamp DESC, source DESC, row id DESC):purchase_history.timestamporpurchase_executions.scheduled_date;purchase_history.idorexecution_id.Both ids are UUIDs and unique per table, so every row has its own position.
purchase_idcan't be the tiebreaker because legacy rows repeat it (migration 000108). Each response listslimitrows of the stream.Keyset cursor, not offset.
has_more: trueandnext_cursor. Passingnext_cursorback ascursorresumes strictly after the last row shown.v1.<source>.<unix µs>.<uuid>. Anything that doesn't decode to that form is a 400, so a bad cursor never reaches SQL.offsetis rejected with 400 instead of being ignored. feat(api): page the History view past the first limit rows (offset, has_more) #674's issue text asked for anoffsetparameter. A client built to that spec would otherwise get the first page back for every offset and keep asking for more.Pending approvals are not paged. The actionable class (pending, notified, scheduled, approved, running, paused) is read and listed the same way as before:
So paging can never push a pending approval off the first page. Before this PR it stayed visible only because each source had its own cap. Now it holds by construction. #686's signals are unchanged: a failed listing or a full actionable class still sets
truncated.Scope.
has_moreis judged on rows the caller may see. The page reads one visible row pastlimit, so rows outside the caller's scope never sethas_moreand never leave a trailing empty page.next_cursoronly names a row that was in the response.filterPurchaseHistoryByAllowedAccounts, which now runs per batch. Its result is mapped back to rows bymarkOutOfScope, so the scope rule still has a single implementation.Store. Two new methods in
internal/config/store_postgres_history_page.go:ListPurchaseHistoryPage:GetPurchaseHistoryFiltered's predicate plus the keyset condition. It also returns each row's id, and still scans through the sharedscanPurchaseHistoryRow.ListHistoryExecutionsPage: the terminal statuses with clean completed rows excluded, thescheduled_datedate bounds pushed into SQL, plus the keyset condition.Each source is read in batches of
max(limit+1, 100), capped at 1000. A batch is read again only when filters or scope hide rows. There is no migration:idx_purchase_history_timestampandidx_purchase_executions_scheduled_datecover both queries.Response and docs.
HistoryResponsegainshas_moreandnext_cursor.truncatedis also true wheneverhas_moreis, so today's banner keeps showing until the UI has Load more.executions_limitnow equalslimit, because terminal executions share the page./api/historynow documents thelimitandcursorparameters, the paging contract, and the new fields.Closes #674
Interaction with #1200 (scope-aware summary totals)
This branch doesn't touch the summary. On its own,
summarystill covers the rows in the response, as it did before. Combined with #1200, the summary is whole-scope on every page.I checked this with a trial merge of this branch and #1200's head (abec6cd) in a scratch worktree:
git merge-treereports conflicts only inhandler_history.goandtypes.go. fix(history): truncation banner quotes the wrong cap and a failed class listing does not set truncated #686 alone already conflicts with feat(api): scope-aware SQL totals for the History summary cards #1200 in both files.getHistory.loadHistoryScopeonce, before the actionable fetch.scope.filter.deferthe sweep withstaleExecsdeclared outside theCursor == nilbranch.historySummary(ctx, filters, scope, all)/SummaryCompleteplus this PR's paging fields.fetchPurchaseHistory. Usefilters.purchaseHistoryFilter()withLimitoverridden by the batch size, then callListPurchaseHistoryPage.TestGetHistory_SummaryFilterMatchesRowFetchFilterandTestGetHistory_SummaryFallsBackToPageWhenAggregateFails. They needGetAllPurchaseHistory/GetPurchaseHistoryFilteredrenamed toListPurchaseHistoryPagewith a thirdmock.Anything(4 lines). Without that change,TestGetHistory_SummaryFilterMatchesRowFetchFilterpanics on a nilrowFilter.Start.go test ./internal/api/passed.TestIntegration_HistorySummary_*passed on real Postgres, and so did this PR'sTestIntegration_HistoryPaging_*.limit=7. Every one of the 18 pages had the identical whole-scope summary, withsummary_complete: true. The pages listed 129 rows in total, exactly thetotal_purchasesthe summary reports.Verification
All on the Linux VM with Go 1.26.9, with
AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true. Integration tests used real Postgres via testcontainers (the shared container). This is local mock and integration evidence only: there's no deployed-stack or real-account coverage. The change is DB and handler only.Fail before, pass after (real Postgres). A throwaway probe seeded 12 completed purchases and paged with
limit=5, followingnext_cursor:"map[p-00 … p-04]" should have 12 item(s), but has 5(nohas_more, no cursor);has_morefalse on the last one, all 12 rows reached.New tests.
internal/api/handler_history_page_integration_test.go, real Postgres:TestIntegration_HistoryPaging_EveryRowOnceInOrder: two tenants, purchases and terminal executions sharing timestamps within and across tables,limit=7. Every row comes back exactly once in stream order. Both pending approvals, older than every other row, are on page 1 and on no later page. A purchase and a failed execution written at the top after page 1 don't shift, repeat or skip anything.TestIntegration_HistoryPaging_ScopedUserNeverPagesIntoAnotherTenant: tenant-B rows are newer, interleaved and older. The tenant-A user gets exactly the tenant-A rows plus their own ambient execution, once each and in order. No cursor names a hidden row, andhas_moreis false on the last page although tenant-B rows follow it. Tenant B's pending approval never appears.internal/config/store_postgres_history_page_integration_test.go, real Postgres:ListPurchaseHistoryPageselects the same rows asGetPurchaseHistoryFilteredfor 8 filter shapes (provider, both account columns, the date edges ±1µs);scheduled_datebounds.internal/api/handler_history_page_test.go, unit/mock:offset, malformed, wrong version, unknown source, out-of-range timestamp, non-UUID, non-canonical UUID, injection, overlong;truncated;Mutation probes. Each one was applied by hand, run, then reverted:
<=→<in the cross-source keyset condition:TestIntegration_HistoryKeyset_CrossSourceTiesandTestIntegration_HistoryPaging_EveryRowOnceInOrderfail.id DESCfrom the purchase_history ORDER BY:TestIntegration_ListPurchaseHistoryPage_WalksEveryRowOncefails.has_morebefore the visibility check:TestHandler_getHistory_ScopedPagingStaysInScopefails. The first version of the test let this survive, so its fixture now ends on a full page.Existing tests. The history handler tests now mock
ListPurchaseHistoryPage/ListHistoryExecutionsPageinstead ofGetAllPurchaseHistory/GetPurchaseHistoryFiltered. For a[]PurchaseHistoryRecordfixture, the shared mock assigns row ids by position, so most of these edits are one-line renames. These assertions changed on purpose:limitrows ⇒ truncated": exactly a page of rows is no longer a false positive, so the tests now uselimit+1.executions_limitnow followslimit.TestHandler_getHistory_ScopedUserTruncationLeaksNoCounts: other tenants' rows no longer settruncated. It still asserts that no count leaks.TestHandler_getHistory_FetchesDisjointClassesWithTheirLimits: asserts one actionable read plus one terminal page read.Combined-tree verification. This branch was tested together with the other queued branches on a local, never-pushed tree:
queue/platform@936996ad= origin/maina14dc8f2plus these heads, merged in this order:fix/issue-72andfix/issue-312conflict on one line ininternal/api/handler_dashboard.go(both change thecalculateCommitmentMetricscall). The tree used #72's side with #312's validatedprovidervariable; whichever PR merges second takes that same resolution.Linux, Go 1.26.9,
AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true; Docker Postgres via testcontainers:936996ad:go build ./...,go vet ./...,go vet -tags integration ./...and the fullgo test -race -short -count=1 ./....936996ad: golangci-lint v2.10.1 found 0 issues on every changed package, with and without--build-tags integration.tests/e2eis a separate module, so it was vetted under its owne2etag.gocyclo -over 10is silent on the changed non-test files.-tags integration -count=1, 60-minute local timeout because the VM was shared):936996ad, these pass:internal/server,internal/credentials,internal/scheduler,internal/database/postgres/migrations,internal/purchaseandinternal/api.internal/configpasses exceptTestPostgresStoreDB_ResolveStaleRIExchange_ValidatesCloudCheck, which fails the same way on unmodified main (fixed by fix(config): compare resolve checked_at at the database's microsecond precision #1252).26795376, same branches at earlier heads):internal/analytics,internal/auth,internal/commitmentopts,internal/databaseandcmd/lambda/clear-rate-limitall pass. The later fixes don't touch those packages.httpToLambdaRequestsignature change;scripts/setup-git-secrets.shpatterns in a fresh repo with an isolated HOME, found nothing in the changed files, and a planted fake key was flagged.scripts/test-git-secrets-allowlist.sh: 137 PASS, 0 FAIL.Notes and things left out
Rows that change state between page loads. A row is listed in the state it has when its page is read. The common case is the stale-approval sweep:
scheduled_dateis at least 7 days old, so a later page normally lists the same execution again, now as expired.The same thing happens to any approval that fails or expires while the user pages. The stream itself never repeats a row; this is one execution appearing once as actionable and once as terminal. The frontend should de-duplicate by
purchase_idand keep the newest load (see below).Scan cost for scoped users. Scope still runs in Go, so filling a page for a user whose rows are sparse can read several batches. In the worst case that's the whole table, as feat(api): scope-aware SQL totals for the History summary cards #1200's aggregate already does, and the date window bounds it when set. Pushing the scope into SQL is fix(history): pending approvals can fall off the first page because filters are applied after the executions cap; add paging and push provider/account/date filters into the executions SQL #632's "PR2" and is not done here.
Scope checks per batch.
markOutOfScopereusesfilterPurchaseHistoryByAllowedAccounts, so a request loads the account scope once per batch instead of once overall. Usually that's 2–3 times, all cheap. With feat(api): scope-aware SQL totals for the History summary cards #1200 merged, this can become oneloadHistoryScopeplusscope.allows.truncatedwording. In types.go and openapi.yaml thetruncateddescription still says it "may be a false positive when exactlylimit… rows exist". That's no longer true for the paged stream. I left that paragraph alone because fix(history): truncation banner quotes the wrong cap and a failed class listing does not set truncated #686 and feat(api): scope-aware SQL totals for the History summary cards #1200 both rewrite it. The newhas_moredescription explains how the two flags relate.Page size semantics.
limitnow counts completed purchases and terminal executions together. Before, the first page could hold up tolimitpurchases plus 100 terminal executions, with no ordering between the two lists. The older of those rows are now on page 2 instead.Predicate copy.
purchaseHistoryPageCondsrepeatsGetPurchaseHistoryFiltered's three trivial conditions and reuses its shared account predicate.TestIntegration_ListPurchaseHistoryPage_MatchesFilteredFetchguards against drift. feat(api): scope-aware SQL totals for the History summary cards #1200 extractspurchaseHistoryFilterWhere; once both are merged, the page query can call it.Test teardown.
internal/apiintegration tests that use the shared container now get it stopped through anafterAllTestshook inmain_nonet_test.go, registered from the newmain_integration_test.go.Lint config.
.golangci.yml: the new store file is added to the existing misspell allowlist for the legacycancelled_bycolumn.Frontend follow-up for the UI owner (#676)
No change under frontend/. What "Load more" needs, in
frontend/src/api/history.ts,frontend/src/history.tsandfrontend/src/types.ts:has_more?: boolean,next_cursor?: stringandactionable_executions_limit?: numbertoHistoryResponse, and letgetHistorypasscursor(andlimitif wanted).has_moreis true, show a "Load more" button under the history table. On click, repeat the request with the same filters pluscursor=next_cursor, then append the returned rows to the history list.has_more: false.purchase_idand keep the row from the newest response. A stale approval swept to expired after page 1 shows up again on a later page in its new state (see the notes above).has_moreis true, the rows can be paged to, so replace "Narrow the date range or filters to see them" with the Load more button. Keep the banner only whentruncated && !has_more, meaning rows that paging can't reach: the actionable cap (quoteactionable_executions_limit) or a failed listing. Theexecutions_limittext in the current banner now equalslimit.Maintainers: please mirror the issue's triage labels on this PR:
priority/p2,severity/medium,urgency/this-quarter,impact/many,effort/m,type/feat,triaged.🤖 Generated with Claude Code