Skip to content

perf: optimize full-text benchmark hot paths#1116

Open
jmk640 wants to merge 3 commits into
oceanbase:vldb_2026from
jmk640:vldb_2026
Open

perf: optimize full-text benchmark hot paths#1116
jmk640 wants to merge 3 commits into
oceanbase:vldb_2026from
jmk640:vldb_2026

Conversation

@jmk640

@jmk640 jmk640 commented Jul 17, 2026

Copy link
Copy Markdown

Summary

  • reuse IK parser arbitration and processor state across tokenization calls, add an ASCII character-classification fast path, and cache constant TOKENIZE() results within the expression context
  • detect predicate-only natural-language MATCH usage and bypass relevance computation when the score is not consumed, while retaining score calculation for projections, comparisons, ordering, shared expressions, and boolean mode
  • cache full-text build expressions and reusable execution state to reduce repeated setup and allocation work during index construction

Motivation

The FTS benchmark spends substantial time in parser setup and token-loop allocation, BM25 work for queries that only need document membership, and repeated expression setup during index construction. These changes target those paths independently while preserving token output, match semantics, and relevance behavior whenever the score is observable.

Validation

  • git diff --check origin/vldb_2026...HEAD
  • added IK parser reuse coverage in unittest/storage/fts/test_ft_parser.cpp
  • changes are split into three independent commits for TOKENIZE, query, and build benchmarking

Benchmarking

Full fts_large_bench.sh before/after measurements are still required on the benchmark environment. Please keep this PR in draft until correctness regression tests and five-run median performance measurements are complete.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@LINxiansheng
LINxiansheng marked this pull request as ready for review July 18, 2026 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants