docs(examples): VolumeCache warm-cache example + transport benchmarks#48
Open
deanq wants to merge 6 commits into
Open
docs(examples): VolumeCache warm-cache example + transport benchmarks#48deanq wants to merge 6 commits into
deanq wants to merge 6 commits into
Conversation
…tive transport Serverless CPU-endpoint benchmark sweeping file-shape quadrants (few/many x small/large + mixed HF) across serial/parallel/tar transport in both mirror and hydrate directions. Provides the empirical basis for the 256 KiB size-bucketed adaptive transport. Includes recorded results.
|
Capy auto-review is paused for this organization because the usage-cycle auto-review limit has been reached. Increase the limit or turn it off in billing settings to resume automatic reviews. |
deanq
changed the title
deanq
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
05_data_workflows/05_volume_warm_cache/example forrunpod.serverless.VolumeCacheplus a benchmark suite, including the quadrant transport benchmark that motivated the adaptive-transport work in runpod-python (#531 → #538).Contents:
gpu_worker.py+ README): a Flash@Endpointthat warm-caches its model via VolumeCache (HF cache on local disk, mirrored to the network volume) instead of pointingHF_HOMEat the mount.benchmark/): direct-on-volume vs VolumeCache cold-start A/B (serverless), arigorous/pod-based harness with page-cache eviction and breakeven analysis, and aquadrant/serverless benchmark.benchmark/quadrant/): a CPU serverless worker that sweeps file-shape quadrants (few/many × small/large + a mixed HF-like tree) across serial/parallel/tar transport in both mirror and hydrate directions, with cold page cache and a byte-for-byte correctness gate. Recorded results inresults.json.Findings (why adaptive transport)
Mirror (local → volume), mean seconds:
Mean file size decides the winner: many-small-file trees favor tar (metadata collapse; 72× over serial), large files favor parallel (tar's single stream is dead weight). This is the empirical basis for the 256 KiB size-bucketed transport in the runpod-python VolumeCache follow-up.
Linear: SLS-367
Test plan
make quality-check(ruff format + lint) passes on the added files.