fix(pyamber): Invalidate cache while loading executor_def. - #7173
Conversation
Backport auto-label reportThis
|
Automated Reviewer SuggestionsBased on the
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 378 | 0.231 | 25,848/38,451/38,451 us | 🔴 +14.4% / 🔴 +137.4% |
| ⚪ | bs=100 sw=10 sl=64 | 781 | 0.477 | 127,291/154,258/154,258 us | ⚪ within ±5% / 🔴 +39.2% |
| 🔴 | bs=1000 sw=10 sl=64 | 912 | 0.557 | 1,096,088/1,244,057/1,244,057 us | 🔴 +10.7% / 🔴 +17.1% |
Baseline details
Latest main 45b7b20 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 378 tuples/sec | 407 tuples/sec | 765.25 tuples/sec | -7.1% | -50.6% |
| bs=10 sw=10 sl=64 | MB/s | 0.231 MB/s | 0.248 MB/s | 0.467 MB/s | -6.9% | -50.5% |
| bs=10 sw=10 sl=64 | p50 | 25,848 us | 22,587 us | 12,703 us | +14.4% | +103.5% |
| bs=10 sw=10 sl=64 | p95 | 38,451 us | 34,502 us | 16,199 us | +11.4% | +137.4% |
| bs=10 sw=10 sl=64 | p99 | 38,451 us | 34,502 us | 19,557 us | +11.4% | +96.6% |
| bs=100 sw=10 sl=64 | throughput | 781 tuples/sec | 771 tuples/sec | 965.22 tuples/sec | +1.3% | -19.1% |
| bs=100 sw=10 sl=64 | MB/s | 0.477 MB/s | 0.471 MB/s | 0.589 MB/s | +1.3% | -19.0% |
| bs=100 sw=10 sl=64 | p50 | 127,291 us | 125,972 us | 103,830 us | +1.0% | +22.6% |
| bs=100 sw=10 sl=64 | p95 | 154,258 us | 162,308 us | 110,778 us | -5.0% | +39.2% |
| bs=100 sw=10 sl=64 | p99 | 154,258 us | 162,308 us | 118,713 us | -5.0% | +29.9% |
| bs=1000 sw=10 sl=64 | throughput | 912 tuples/sec | 927 tuples/sec | 992.33 tuples/sec | -1.6% | -8.1% |
| bs=1000 sw=10 sl=64 | MB/s | 0.557 MB/s | 0.566 MB/s | 0.606 MB/s | -1.6% | -8.0% |
| bs=1000 sw=10 sl=64 | p50 | 1,096,088 us | 1,077,186 us | 1,016,959 us | +1.8% | +7.8% |
| bs=1000 sw=10 sl=64 | p95 | 1,244,057 us | 1,124,134 us | 1,062,501 us | +10.7% | +17.1% |
| bs=1000 sw=10 sl=64 | p99 | 1,244,057 us | 1,124,134 us | 1,082,611 us | +10.7% | +14.9% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,528.70,200,128000,378,0.231,25848.21,38450.79,38450.79
1,100,10,64,20,2561.40,2000,1280000,781,0.477,127291.16,154257.67,154257.67
2,1000,10,64,20,21923.56,20000,12800000,912,0.557,1096087.94,1244057.08,1244057.08
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7173 +/- ##
============================================
+ Coverage 79.27% 79.37% +0.10%
Complexity 3787 3787
============================================
Files 1160 1160
Lines 46124 46125 +1
Branches 5114 5114
============================================
+ Hits 36563 36610 +47
+ Misses 7942 7896 -46
Partials 1619 1619
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Backport PR opened: draft #7174 (#7174) to |
What changes were proposed in this PR?
This PR calls
importlib.invalidate_caches()prior to importing newly written UDF modules inExecutorManager.load_executor_definition.Any related issues, documentation, discussions?
Fixes #7126
How was this PR tested?
Tested with existing Python unit tests:
amber/src/test/python/core/architecture/managers/test_executor_manager.py::TestUpdateExecutor::test_repeated_updates_keep_carrying_the_running_stateVerified that rapid sequential executor updates cleanly load without throwing
ModuleNotFoundError.Was this PR authored or co-authored using generative AI tooling?
No.