perf: small-integer Val.Num cache for arithmetic fast paths#683
Open
He-Pin wants to merge 1 commit intodatabricks:masterfrom
Open
perf: small-integer Val.Num cache for arithmetic fast paths#683He-Pin wants to merge 1 commit intodatabricks:masterfrom
He-Pin wants to merge 1 commit intodatabricks:masterfrom
Conversation
Add a 256-entry pool of pre-allocated Val.Num instances for integers 0-255 in Val.cachedNum(). Applied to all evaluator arithmetic operations (+, -, *, /, %, unary, bitwise, shift) and base64DecodeBytes. Changes: - Val.scala: Add numCache array[256] and cachedNum() factory method that returns cached instance for small non-negative integers, fresh otherwise - Evaluator.scala: Replace Val.Num(pos, ...) with Val.cachedNum(pos, ...) in all arithmetic paths (binary +/-/*/div/mod, unary +/-/~, shift, bitwise) - EncodingModule.scala: Use cachedNum in base64DecodeBytes (byte values 0-255 are always cache hits), also convert .map to while-loop JMH improvements (ms/op): - base64DecodeBytes: 9.423 → 8.717 (-7.5%) - bench.03 (fibonacci): 13.399 → 12.473 (-6.9%) - bench.02: 46.694 → 44.034 (-5.7%) - realistic2: 70.491 → 67.856 (-3.7%) - Zero regressions across 35 benchmarks Native (hyperfine): base64DecodeBytes 41.2ms → 38.7ms (-6.1%) Upstream: he-pin/sjsonnet jit branch commit a59223a
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Jsonnet programs frequently produce small integer results (0, 1, 2) from arithmetic, array indexing, and loop counters. Each occurrence currently allocates a fresh
Val.Numheap object.Key Design Decision
Cache
Val.Numinstances for integers 0–255 in a pre-allocated array. WhenNum.apply(v)is called andvis an integer in this range, return the cached instance. This eliminates millions of micro-allocations in loop-heavy workloads without affecting semantics.Modification
Val.Numcache array initialized at class load timeVal.Num.applyto check the cache before allocatingBenchmark Results
JMH (JVM, 3 iterations warmup + 3 measurement)
Analysis
The small-integer cache is a well-known optimization pattern (used in Java Integer.valueOf, Python small ints). The tight error bars on the PR results (±2.5 vs master ±38.9) suggest more stable GC behavior due to reduced allocation pressure. The comparison2 benchmark benefits most (-19.3%) due to heavy loop/arithmetic workload.
References
a59223afResult
All tests pass. All benchmarks positive, no regressions.