Skip to content

feat(query): add RANK for scoring hyperedges against a query hyperedge - #109

Merged
venkat1701 merged 6 commits into
mainfrom
feature/rank-statement
Oct 27, 2026
Merged

venkat1701 merged 6 commits into
mainfrom
feature/rank-statement

Conversation

@venkat1701

@venkat1701 venkat1701 commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #108 (uses its role-filtered posting cursors). Merge #108 first; this PR then retargets to main.

Adds role- and time-aware scoring of hyperedges as a new statement. MATCH is unchanged, so no existing query changes behaviour.

Statement

RANK EDGE e:Review
  LIKE ($alice AS actor REQUIRED, $bob AS reviewer, ANY Document AS target)
  VALID AT 100 TAU 100
  PREFER e.status = 'approved'
  SIZE 3
  TOP 3;

Output: edge, score, members, roles, attributes, time, topology. Each part is between 0 and 1 with a fixed meaning, so scores are comparable across queries; parts the query does not use are NULL and get weight 0. Full reference in docs/database/hql.md#rank, algorithm and costs in the new docs/database/ranking.md.

How it runs

  • REQUIRED named slots give the candidates: the intersection of their reverse-index postings, role-filtered. Without one, every hyperedge of the type is a candidate, so the answer stays exact.
  • Each candidate gets an upper bound (named slots exact via one incidence lookup each, typed slots and preferences counted as perfect, size exact). Candidates are fully scored in descending bound order until the next bound is below the k-th score. Ties go to the lower id, the same as sorting every score.
  • New package io.hstore.db.retrieve: RankQuery, RoleTimeScorer, Ranking.

Limits and safety

  • TOP up to 10,000, at most 16 slots and 16 preferences (checked while parsing), weights, TAU, SCALE and intervals validated.
  • Candidates, full scores and member reads for typed slots take rows of Budget.DEFAULT (1,000,000 rows, 30 s); page reads use the statement's page budget. Going over fails the statement rather than returning a partial ranking. The per-query member-type cache is capped.
  • Named atoms resolve through Reader.require, so another tenant's atom fails exactly like a missing one; typed slots only count members Reader.atom returns; candidates are admitted through Reader.atom before anything else is read.
  • RANK is logged as a read by the statement logger.
  • Small fix on the way: an empty interval ([5, 5)) in any statement was an internal IllegalArgumentException; it is now a syntax error.

Tests

  • RankTest.workedExampleRanksByTheFiveComponents: the example from the design doc, scores checked to 1e-9.
  • RankTest.earlyStoppingAgreesWithScoringEverything: 300 random queries (random slots, roles, required flags, time specs, preferences, sizes and weights) over 500 random edges; checks that the bound is never below the score for every edge, and that the early-stopping top k equals scoring every edge. Making the bound too tight for typed slots makes it fail.
  • Required typed slots, untyped queries, repeated atoms, budget exhaustion, 25 invalid statements, NaN and infinite property values, time arithmetic at the ends of the long range.
  • SecurityTest.rankStaysInsideTheTenant: another tenant's atom cannot be named, and a cross-tenant member does not fill a typed slot.
  • Full build: engine 257, database 105, server 28, all passing.

An independent review found no correctness issues in the bound or the early stop; its smaller findings (NaN scores, exact time arithmetic above 2^53, the unbounded member-type cache, caps checked after slot resolution, the statement-logger category) are fixed here.

RANK EDGE e:T LIKE (slots) [VALID AT t [TAU d] | VALID DURING [f, u)]
[PREFER ...] [SIZE n] [WEIGHTS (...)] TOP k returns the k hyperedges that
best fit a query hyperedge. The score is a weighted sum of five parts in
[0, 1]: members, roles, attributes, time and topology. MATCH is
unchanged.

REQUIRED named slots drive the candidates through role-filtered posting
cursors; otherwise every hyperedge of the type is a candidate. Each
candidate gets an upper bound from cheap reads, and candidates are fully
scored in bound order until no remaining bound can enter the top k, which
returns exactly what scoring every candidate would.

Candidates, full scores and member reads are charged to a row budget,
slots and preferences are capped while parsing, and an empty interval in
any statement is now a syntax error instead of an internal one.
@venkat1701
venkat1701 changed the base branch from feature/role-filtered-postings to main October 27, 2026 12:27
@venkat1701
venkat1701 merged commit ca21405 into main Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant