Skip to content

fix(parsers): key query cache by Language object to prevent id reuse collisions#645

Merged
vitali87 merged 2 commits into
mainfrom
fix/query-cache-language-key
Jul 7, 2026
Merged

fix(parsers): key query cache by Language object to prevent id reuse collisions#645
vitali87 merged 2 commits into
mainfrom
fix/query-cache-language-key

Conversation

@vitali87

@vitali87 vitali87 commented Jul 7, 2026

Copy link
Copy Markdown
Owner

Problem

The JS/TS assignment-function ingestion tests (test_assignment_arrow_functions_are_ingested, test_object_literal_methods_are_ingested, test_export_const_function_expression, ...) flake in CI, almost exclusively on py3.13 jobs, across all three OSes. This is what failed Unit Tests (ubuntu-latest, py3.13) on #643 and several other recent PR runs.

Root cause

get_cached_query keyed its module-level cache on (id(language_obj), query_text) without retaining the Language wrapper. Query does not hold a Python reference to its Language either (refcount stays at baseline after construction). So once a test's wrapper is garbage collected, a later Language allocation (possibly for a different grammar) can land at the same address, and the cache serves a Query compiled for the wrong grammar. Captures silently come back empty (the code path swallows the error into a debug log), and the assignment-arrow tests assert False.

Whether an address gets reused depends on allocator and GC timing, which is why the failures are nondeterministic and skew toward py3.13.

Fix

Key the cache on the Language object itself. Language.__hash__/__eq__ follow the underlying grammar pointer, so distinct wrappers of the same grammar share one cache entry, the dict pins the key alive, and an address can never be reused into a stale hit.

RED: test_query_cache.py::test_cache_keyed_by_grammar_not_wrapper_address fails on main (two wrappers of the same grammar get separate cache entries). GREEN with the one-line key change.

Full non-integration suite: 4399 passed, 5 skipped.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

@greptile-apps

greptile-apps Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes tree-sitter query caching for parser queries. The main changes are:

  • Cache keys now use the Language object with the query text instead of id(language_obj).
  • The cache now retains the Language key, avoiding stale hits after wrapper garbage collection and address reuse.
  • A regression test checks that separate wrappers for the same JavaScript grammar share one cached query.
  • uv.lock is updated to the package version 0.0.246.

Confidence Score: 5/5

Safe to merge with minimal risk.

The production change is small and directly addresses the stale query cache failure mode. The new test covers the intended same-grammar wrapper cache behavior. No blocking correctness or security issues were identified.

No files require special attention.

T-Rex T-Rex Logs

What T-Rex did

  • Reviewed the contract-validation proof to confirm that the blocked pytest attempt and the harness run were captured in the validation log.
  • Executed the generated harness to run the changed utility while bypassing broad optional dependencies.
  • Verified the final harness command exited with code 0 and produced HARNESS_RESULT: PASS.
  • Inspected the harness output showing cache size after second get_cached_query: 1 and that returned Query objects identical: True.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
codebase_rag/parsers/utils.py Updates the query cache key to use Language object equality and the query text.
codebase_rag/tests/test_query_cache.py Adds regression coverage for cache reuse across wrappers of the same grammar.
uv.lock Updates the editable package version entry to 0.0.246.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Caller
participant Cache as get_cached_query
participant Last as _QUERY_LAST
participant Dict as _QUERY_CACHE
participant TS as tree_sitter.Query

Caller->>Cache: Language object + query_text
Cache->>Cache: "key = (language_obj, query_text)"
Cache->>Last: compare last key
alt last key matches
    Last-->>Cache: cached Query
else no last hit
    Cache->>Dict: lookup key
    alt cache miss
        Cache->>TS: Query(language_obj, query_text)
        TS-->>Dict: store Query under Language key
    end
    Dict-->>Cache: cached Query
    Cache->>Last: update last key/result
end
Cache-->>Caller: Query
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Caller
participant Cache as get_cached_query
participant Last as _QUERY_LAST
participant Dict as _QUERY_CACHE
participant TS as tree_sitter.Query

Caller->>Cache: Language object + query_text
Cache->>Cache: "key = (language_obj, query_text)"
Cache->>Last: compare last key
alt last key matches
    Last-->>Cache: cached Query
else no last hit
    Cache->>Dict: lookup key
    alt cache miss
        Cache->>TS: Query(language_obj, query_text)
        TS-->>Dict: store Query under Language key
    end
    Dict-->>Cache: cached Query
    Cache->>Last: update last key/result
end
Cache-->>Caller: Query
Loading

Reviews (2): Last reviewed commit: "test(parsers): mark query cache regressi..." | Re-trigger Greptile

Comment thread codebase_rag/tests/test_query_cache.py Outdated
@codecov-commenter

codecov-commenter commented Jul 7, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 90.47619% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
codebase_rag/tests/test_query_cache.py 87.50% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@vitali87

vitali87 commented Jul 7, 2026

Copy link
Copy Markdown
Owner Author

@greptile review

@sonarqubecloud

sonarqubecloud Bot commented Jul 7, 2026

Copy link
Copy Markdown

@vitali87 vitali87 merged commit 0fe2dbc into main Jul 7, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants