Skip to content

feat(benchmark): M1 IMU 高频摄入压测(引擎层)+ 诚实结果#124

Merged
NeverENG merged 2 commits into
mainfrom
feat/m1-ingest-benchmark
Jun 6, 2026
Merged

feat(benchmark): M1 IMU 高频摄入压测(引擎层)+ 诚实结果#124
NeverENG merged 2 commits into
mainfrom
feat/m1-ingest-benchmark

Conversation

@NeverENG
Copy link
Copy Markdown
Owner

@NeverENG NeverENG commented Jun 6, 2026

M1 第一步:用开环压测验证存储引擎能否扛住高频传感器摄入。落点 benchmark/ingest/(独立 package,不动现有 benchmark/)+ docs-step/ 方案与结果。

做了什么

  • 饱和相(闭环打满)找峰值写速率;开环相(定速率、有界队列、满即丢帧)证定速率丢帧/尾延迟/内存。

诚实结果(经 60s 复测修正)

  • 峰值写速率 ~2.0M w/s,但打满时内存无界(heap 1.5GiB)。
  • 内存在 50k–200kHz 下都随时间持续增长,60s 无平稳态(短跑 6s 的"内存有界"是错觉)。
  • 丢帧经核为队列深度/Windows 计时毛刺,非引擎极限(500kHz 大队列下丢帧归零)。

抓到的真问题

MemTable 写入无背压StartFlush 非阻塞、Put 永不阻塞)+ SSTable/bloom 元数据累积 → "内存封顶"在持续负载下不成立。下一项存储改进:加写背压,再附"前后内存随时间曲线"。

详见 docs-step/M1-ingest-benchmark-result.md

🤖 Generated with Claude Code

NeverENG and others added 2 commits June 6, 2026 22:20
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- 新增 benchmark/ingest:饱和相(找吞吐天花板) + 开环相(定速率证丢帧/尾延迟/内存)
- 发现:MemTable 写入无背压,持续负载下内存随时间无界增长(60s 复测推翻短跑'内存有界'结论)
- 丢帧经核为队列深度/计时毛刺,非引擎极限

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 6, 2026

Warning

Review limit reached

@NeverENG, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 4 minutes and 58 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b69eced2-3d9e-40ef-8ab6-1eb3eee4fb14

📥 Commits

Reviewing files that changed from the base of the PR and between fb0d53c and 3549336.

📒 Files selected for processing (3)
  • benchmark/ingest/main.go
  • docs-step/M1-ingest-benchmark-plan.md
  • docs-step/M1-ingest-benchmark-result.md
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/m1-ingest-benchmark

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 6, 2026

🐯 BanGD 数据库内核评审

整体风险:🟢 低

变更总结:该 PR 新增 benchmark/ingest/ 压测程序,用于在引擎层(A1)验证 BanDB 存储引擎能否扛住高频传感器摄入。核心设计是开环(open-loop)模型:生产者以固定速率通过有界 channel 投递样本、队列满即丢帧;饱和相(闭环打满)测峰值写吞吐。PR 同时产出 docs-step/M1-ingest-benchmark-plan.md(方案说明)和 docs-step/M1-ingest-benchmark-result.md(诚实结果)。PR 不动存储引擎本身的代码,仅扩展压测框架,但结果文档抓到了引擎层的一个关键架构问题——MemTable 写入无背压、Put 永不阻塞,导致内存在持续负载下无界增长。

本评审不阻塞合入;架构级建议以 Issue 形式跟踪,普通问题在下方内联列出。

未发现需要从架构层面改进的问题。


本次评审消耗 token:共 37477 tokens(输入 37135,输出 342,缓存命中 0,缓存写入 0)|维度 [concurrency, memory, storage, performance, resource]|补充阅读周边文件 [storage/engine.go, storage/zstorage/memtable.go]|对抗式复核 3 票/条,过滤疑似误报 0 条

@NeverENG NeverENG merged commit 0c1e1af into main Jun 6, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant