Skip to content

feat: ESO gain optimization (2nd-order LESO, argmin GSS, at_ceiling marker)#136

Open
nerdCopter wants to merge 2 commits into
masterfrom
20260402_Quickflash-ESO
Open

feat: ESO gain optimization (2nd-order LESO, argmin GSS, at_ceiling marker)#136
nerdCopter wants to merge 2 commits into
masterfrom
20260402_Quickflash-ESO

Conversation

@nerdCopter

@nerdCopter nerdCopter commented Apr 2, 2026

Copy link
Copy Markdown
Owner

AI Generated pull-request

Summary

Adds offline 2nd-order LESO (Linear Extended State Observer) bandwidth optimization from blackbox flight data, with time-domain output plots and markdown report integration.

CLI Flags

  • --eso — enables ESO gain optimization (disabled by default)
  • --eso-b0 <value> — overrides control effectiveness b0 (default: auto-estimated via OLS on high-excitation samples using the QuickFlash method)

New Files

src/eso.rs

Core optimizer. Per axis:

  • Extracts filtered gyro (omega) and PID sum (P+I+D+F) as measured output and control input
  • Simulates a discrete Euler-forward 2nd-order LESO at each candidate omega_0:
    • e = omega_meas[k] - omega_hat
    • omega_hat += Ts * (f_hat + b0 * u[k] + beta1 * e)
    • f_hat += Ts * (beta2 * e)
  • Bandwidth parameterisation (Gao 2003): beta1 = 2*omega_0, beta2 = omega_0^2
  • Minimises MSE(omega_hat, omega_meas) via golden-section search over [ESO_OMEGA0_MIN, min(sample_rate/3, ESO_OMEGA0_MAX)]
  • Stability constraint: omega_0 < sample_rate / 3 (enforced automatically)
  • at_ceiling: bool set when the optimizer reaches the stability-constrained upper bound
  • b0 auto-estimated via OLS on rate derivatives; falls back to ESO_DEFAULT_B0 when too few high-excitation samples are available
  • Returns EsoResult per axis: omega0_opt, beta1, beta2, b0, b0_auto, mse, at_ceiling

src/plot_functions/plot_eso.rs

Stacked time-domain output plot per axis:

  • omega_meas (blue), omega_hat (orange), f_hat (green, scaled to ESO_FHAT_Y_FRACTION of Y half-range)
  • Legend shows [at ceiling] when optimizer hit its upper bound
  • Output: <root>_ESO_output_stacked.png

Modified Files

src/main.rs

  • --eso / --eso-b0 flag parsing and validation
  • ESO block runs before report generation so eso_results and the ESO PNG link are available to FlightReport
  • png_links.push(...) for the ESO plot on successful write
  • eso_results passed into FlightReport

src/report.rs

  • eso_results: [Option<EsoResult>; AXIS_COUNT] field on FlightReport
  • ESO Gain Optimization section in generate_markdown_report(): table with omega_0, beta1, beta2, b0, b0 source, MSE, and [at ceiling] note per axis; section is omitted when ESO was not run

src/constants.rs

ESO tuning and plot constants: ESO_OMEGA0_MIN/MAX, ESO_GSS_TOLERANCE/MAX_ITER, ESO_DEFAULT_B0, ESO_N_AHEAD_STEPS, ESO_WARMUP_FRACTION, ESO_B0_* thresholds, ESO_OMEGA0_STABILITY_RATIO, ESO_FHAT_Y_FRACTION, COLOR_ESO_MEAS/HAT/FHAT

OVERVIEW.md

ESO section and TOC entry

Cargo.toml / Cargo.lock

Adds argmin 0.11 dependency

@coderabbitai

coderabbitai Bot commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds an optional 2nd-order LESO bandwidth optimizer and optional Markdown statistical reporting, new eso and report modules with plotting support, CLI flags (--eso, --eso-axis, --eso-b0, --report), new ESO constants/colors, per-axis ESO execution in main, and an argmin dependency.

Changes

Cohort / File(s) Summary
Documentation
OVERVIEW.md
Documented ESO flags (--eso, --eso-axis, --eso-b0), --report output and <stem>_report.md; added sections describing src/eso.rs behavior and src/report.rs output; listed *_report.md among generated outputs.
Constants
src/constants.rs
Added ESO tuning/search constants (ESO_OMEGA0_MIN, ESO_OMEGA0_MAX, ESO_GSS_TOLERANCE, ESO_GSS_MAX_ITER, ESO_DEFAULT_B0, ESO_N_AHEAD_STEPS, ESO_WARMUP_FRACTION, ESO_B0_MIN_CONTROL_THRESHOLD, ESO_B0_MIN_OLS_SAMPLES, ESO_B0_ESTIMATE_MIN_POSITIVE, ESO_FHAT_Y_FRACTION) and plot color aliases (COLOR_ESO_MEAS, COLOR_ESO_HAT, COLOR_ESO_FHAT).
ESO module
src/eso.rs
New module implementing per-axis 2nd-order LESO bandwidth optimization. Adds EsoConfig, EsoResult, and run_eso_optimization(...) which validates inputs, optionally auto-estimates b0 (OLS), constrains search by sample rate, performs golden-section search over omega0 minimizing N-step-ahead MSE, computes gains, and returns metrics and traces.
Report module
src/report.rs
New module to generate Markdown statistical reports. Adds SignalStats, compute_signal_stats, and generate_markdown_report(...) to produce metadata, filtered firmware/config section, per-axis signal statistics tables, and per-axis ESO result sections when provided; writes <stem>_report.md.
Public API
src/lib.rs
Exported new submodules pub mod eso; and pub mod report;.
CLI & Main
src/main.rs
Extended PlotConfig with ESO/report fields, added CLI parsing for --eso, --eso-axis, --eso-b0, --report; process_file() runs run_eso_optimization per selected axis (requires sample_rate), collects EsoResults, optionally produces ESO plot and <stem>_report.md, and preserves ESO/report flags when resetting plot-only flags.
Plotting
src/plot_functions/mod.rs, src/plot_functions/plot_eso.rs
Exported plot_eso module and added plot_eso_output(...) which aligns/truncates ESO traces, computes unified Y scale and optional f_hat scaling, and draws annotated stacked plots per axis (omega_hat, omega_meas, optional f_hat).
Data input tweak
src/data_input/pid_metadata.rs
Refactored INAV-style FF assignment to a conditional expression (Some(values[3]) when > 0), preserving prior behavior.
Build / Dependencies
Cargo.toml
Added dependency argmin = "0.11" for golden-section search optimization.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title accurately describes the main feature additions: ESO gain optimization with 2nd-order LESO and argmin-based golden-section search, plus mentions the at_ceiling marker enhancement.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 20260402_Quickflash-ESO

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot]

This comment was marked as outdated.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (5)
src/main.rs (1)

1316-1331: Minor: any_valid guard is effectively unreachable.

Any unknown/empty axis token already triggers print_usage_and_exit via the other arm, so the if any_valid check at line 1329 never fires for the false case. It's harmless, but you could drop it and just always assign plot_config.eso_axes = axes; for clarity. Not a bug.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/main.rs` around lines 1316 - 1331, The any_valid boolean guard is
redundant because unknown tokens call print_usage_and_exit in the match's other
arm; remove the any_valid flag and its conditional and always assign
plot_config.eso_axes = axes after parsing. Specifically, delete the any_valid
variable and the if any_valid { ... } check, leaving axes to be set directly
into plot_config.eso_axes (referencing the axes array, the any_valid variable,
and plot_config.eso_axes and the print_usage_and_exit call to locate the code).
src/report.rs (2)

176-176: Use AXIS_COUNT instead of AXIS_NAMES.len().min(3).

axis_names.rs already defines AXIS_COUNT = 3 with a compile-time assertion that it equals AXIS_NAMES.len(). The .min(3) embeds a magic number that duplicates that invariant.

-    for axis_idx in 0..AXIS_NAMES.len().min(3) {
+    for axis_idx in 0..crate::axis_names::AXIS_COUNT {

As per coding guidelines: "Axis indices (0=Roll, 1=Pitch, 2=Yaw) must use the src/axis_names.rs module with AXIS_COUNT, AXIS_NAMES, and axis_name() function instead of creating redundant axis index constants".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/report.rs` at line 176, The loop uses a magic-number expression
AXIS_NAMES.len().min(3); replace that with the canonical constant AXIS_COUNT
(from axis_names.rs) so axis indexing relies on the single authoritative value;
update the for loop range to for axis_idx in 0..AXIS_COUNT and ensure any
related uses rely on AXIS_NAMES and axis_name() rather than duplicated
constants.

78-82: Dead condition and magic number.

The outer loop iterates 0..AXIS_NAMES.len().min(3) (= 0..3), so axis < 4 is always true and the check is dead. If the intent was to also report Throttle (setpoint index 3), the loop bound needs to change; otherwise drop the guard. The literal 4 is also a magic number per the repo guideline.

-        if axis < 4 {
-            if let Some(sp) = row.setpoint[axis] {
-                setpoint.push(sp);
-            }
+        if let Some(sp) = row.setpoint[axis] {
+            setpoint.push(sp);
         }

As per coding guidelines: "All constants go in src/constants.rs — no hardcoded magic numbers in function code".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/report.rs` around lines 78 - 82, The condition `if axis < 4` in the block
using `axis` and `row.setpoint[axis]` is dead because the loop already iterates
`0..AXIS_NAMES.len().min(3)`; either extend the loop bound to include Throttle
(index 3) or remove the guard entirely. Replace the hardcoded `4` with a named
constant (e.g., MAX_REPORTED_AXIS) defined in src/constants.rs and use that
constant to control the loop or the check; update the loop range to
`0..AXIS_NAMES.len().min(MAX_REPORTED_AXIS)` if you intend to include index 3,
otherwise delete the `axis < ...` guard. Ensure references are to `AXIS_NAMES`,
the loop variable `axis`, and `row.setpoint` so the change is localized to the
existing logic.
src/eso.rs (1)

244-253: Minor: move the / 3.0 stability divisor into constants.rs.

The sample_rate / 3.0 stability bound is a domain-specific constant (LESO discrete-time stability limit) repeated inline. Putting it in src/constants.rs (e.g. ESO_STABILITY_DIVISOR: f64 = 3.0) keeps the rule in one place and matches how the other ESO knobs are defined.

As per coding guidelines: "All constants go in src/constants.rs — no hardcoded magic numbers in function code".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/eso.rs` around lines 244 - 253, Replace the hardcoded divisor 3.0 with a
named constant defined in src/constants.rs (e.g. ESO_STABILITY_DIVISOR: f64 =
3.0) and use that constant in the omega0_max_stable calculation in the function
that computes omega0_max_stable (references: omega0_max_stable, sample_rate,
config.omega0_max, config.omega0_min); update the error message computation that
multiplies config.omega0_min * 3.0 to use the same constant (config.omega0_min *
ESO_STABILITY_DIVISOR) so the stability rule is centralized in constants.rs.
src/plot_functions/plot_eso.rs (1)

51-54: Nit: p95_idx name is misleading.

The index is computed from UNIFIED_Y_AXIS_PERCENTILE, not a hardcoded 95th percentile. Rename to pctl_idx (or similar) so it doesn't imply a specific value if the constant is later tuned.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/plot_functions/plot_eso.rs` around lines 51 - 54, The local variable name
p95_idx is misleading because the index is derived from
UNIFIED_Y_AXIS_PERCENTILE (a tunable constant); rename p95_idx to a neutral name
like pctl_idx (or percentile_idx) in the block that computes half_range and
update any subsequent uses in the same scope to match (referencing all_abs,
UNIFIED_Y_AXIS_PERCENTILE, UNIFIED_Y_AXIS_HEADROOM_SCALE,
UNIFIED_Y_AXIS_MIN_SCALE and the half_range calculation) so the code reflects
the generic percentile-based computation.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/plot_functions/plot_eso.rs`:
- Around line 98-108: The f_hat scaling uses UNIFIED_Y_AXIS_MIN_SCALE instead of
the actual plot half_range, so compute fhat_scale after half_range is known
(i.e., inside draw_stacked_plot or by passing half_range into where fhat_scale
is built) and base it on data.fhat_max_abs and half_range so f_hat fills the
intended fraction of the Y range; replace the magic 0.5 with a new constant
ESO_FHAT_Y_FRACTION placed in src/constants.rs, and update usages of
fhat_scale/fhat_max_abs in plot_eso.rs (references: build_axis_data,
draw_stacked_plot, data.fhat_max_abs, fhat_scale, UNIFIED_Y_AXIS_MIN_SCALE) to
use the new calculation and constant.

---

Nitpick comments:
In `@src/eso.rs`:
- Around line 244-253: Replace the hardcoded divisor 3.0 with a named constant
defined in src/constants.rs (e.g. ESO_STABILITY_DIVISOR: f64 = 3.0) and use that
constant in the omega0_max_stable calculation in the function that computes
omega0_max_stable (references: omega0_max_stable, sample_rate,
config.omega0_max, config.omega0_min); update the error message computation that
multiplies config.omega0_min * 3.0 to use the same constant (config.omega0_min *
ESO_STABILITY_DIVISOR) so the stability rule is centralized in constants.rs.

In `@src/main.rs`:
- Around line 1316-1331: The any_valid boolean guard is redundant because
unknown tokens call print_usage_and_exit in the match's other arm; remove the
any_valid flag and its conditional and always assign plot_config.eso_axes = axes
after parsing. Specifically, delete the any_valid variable and the if any_valid
{ ... } check, leaving axes to be set directly into plot_config.eso_axes
(referencing the axes array, the any_valid variable, and plot_config.eso_axes
and the print_usage_and_exit call to locate the code).

In `@src/plot_functions/plot_eso.rs`:
- Around line 51-54: The local variable name p95_idx is misleading because the
index is derived from UNIFIED_Y_AXIS_PERCENTILE (a tunable constant); rename
p95_idx to a neutral name like pctl_idx (or percentile_idx) in the block that
computes half_range and update any subsequent uses in the same scope to match
(referencing all_abs, UNIFIED_Y_AXIS_PERCENTILE, UNIFIED_Y_AXIS_HEADROOM_SCALE,
UNIFIED_Y_AXIS_MIN_SCALE and the half_range calculation) so the code reflects
the generic percentile-based computation.

In `@src/report.rs`:
- Line 176: The loop uses a magic-number expression AXIS_NAMES.len().min(3);
replace that with the canonical constant AXIS_COUNT (from axis_names.rs) so axis
indexing relies on the single authoritative value; update the for loop range to
for axis_idx in 0..AXIS_COUNT and ensure any related uses rely on AXIS_NAMES and
axis_name() rather than duplicated constants.
- Around line 78-82: The condition `if axis < 4` in the block using `axis` and
`row.setpoint[axis]` is dead because the loop already iterates
`0..AXIS_NAMES.len().min(3)`; either extend the loop bound to include Throttle
(index 3) or remove the guard entirely. Replace the hardcoded `4` with a named
constant (e.g., MAX_REPORTED_AXIS) defined in src/constants.rs and use that
constant to control the loop or the check; update the loop range to
`0..AXIS_NAMES.len().min(MAX_REPORTED_AXIS)` if you intend to include index 3,
otherwise delete the `axis < ...` guard. Ensure references are to `AXIS_NAMES`,
the loop variable `axis`, and `row.setpoint` so the change is localized to the
existing logic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 209c4fdb-6a38-4c19-8c97-10636f7e228e

📥 Commits

Reviewing files that changed from the base of the PR and between 5315727 and 4105394.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (7)
  • Cargo.toml
  • src/constants.rs
  • src/eso.rs
  • src/main.rs
  • src/plot_functions/mod.rs
  • src/plot_functions/plot_eso.rs
  • src/report.rs
✅ Files skipped from review due to trivial changes (3)
  • Cargo.toml
  • src/plot_functions/mod.rs
  • src/constants.rs

Comment thread src/plot_functions/plot_eso.rs
coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

@nerdCopter

Copy link
Copy Markdown
Owner Author

@coderabbitai, all nitpicks resolved? full review

@coderabbitai

This comment was marked as outdated.

@nerdCopter

This comment was marked as outdated.

coderabbitai[bot]
coderabbitai Bot previously approved these changes May 11, 2026
@coderabbitai

This comment was marked as outdated.

@nerdCopter

Copy link
Copy Markdown
Owner Author

@Quick-Flash is this useful/working? still interested?

@nerdCopter

This comment was marked as outdated.

@coderabbitai

This comment was marked as outdated.

@nerdCopter nerdCopter changed the title Add ESO gain optimization + report generation Add ESO gain optimization Jun 10, 2026
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jun 10, 2026
@nerdCopter

nerdCopter commented Jun 10, 2026

Copy link
Copy Markdown
Owner Author

@Quick-Flash , is this ESO thing functioning as desired?

[edit: replaced graphs with better legend]

EMUF_BLACKBOX_LOG_HELIO_V1_20260503_140158_master_43136752e1 01_ESO_output_stacked EMUF_BLACKBOX_LOG_HELIO_V1_20260503_140158_master_43136752e1 02_ESO_output_stacked

via
EMUF_BLACKBOX_LOG_HELIO_V1_20260503_140158_master_43136752e1.zip

Add 2nd-order LESO (Linear Extended State Observer) bandwidth optimizer:

- src/eso.rs: golden-section search via argmin over omega_0 ∈ [ESO_OMEGA0_MIN,
  min(sample_rate/ESO_OMEGA0_STABILITY_RATIO, ESO_OMEGA0_MAX)].  N-step-ahead
  open-loop prediction cost (unimodal: low omega_0 = stale f_hat, high = noise).
  b0 auto-estimated via OLS on rate derivatives (QuickFlash method); falls back
  to ESO_DEFAULT_B0 when too few high-excitation samples.
  at_ceiling: bool set when omega0_opt >= omega0_max_stable - ESO_GSS_TOLERANCE.
- src/plot_functions/plot_eso.rs: stacked time-domain ESO output plot per axis
  (omega_meas blue, omega_hat orange, f_hat green scaled to ESO_FHAT_Y_FRACTION).
  Legend shows [at ceiling] when optimizer hit its stability-constrained bound.
- src/constants.rs: ESO_OMEGA0_MIN/MAX, GSS_TOLERANCE, GSS_MAX_ITER,
  DEFAULT_B0, N_AHEAD_STEPS, WARMUP_FRACTION, B0_* thresholds,
  OMEGA0_STABILITY_RATIO, FHAT_Y_FRACTION, COLOR_ESO_*.
- src/main.rs: --eso flag (eso_requested bool); --eso-b0 sets b0 with
  eso_b0_user_override; plot_config.run_eso/eso_b0/eso_b0_user_override
  applied after arg loop; --eso alone triggers PlotConfig::none() so only
  ESO output is generated.
- Cargo.toml/Cargo.lock: add argmin 0.11 dependency.
- OVERVIEW.md: ESO section and TOC entry.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@nerdCopter nerdCopter force-pushed the 20260402_Quickflash-ESO branch from eab71ab to ccaad92 Compare June 11, 2026 18:58
@nerdCopter nerdCopter changed the title Add ESO gain optimization feat: ESO gain optimization (2nd-order LESO, argmin GSS, at_ceiling marker) Jun 11, 2026
- Merge origin/master (fddd046, PR #156 always-on markdown report)
- Add eso_results: [Option<EsoResult>; AXIS_COUNT] field to FlightReport
- Add ESO Gain Optimization section to generate_markdown_report:
  table of omega_0, beta1, beta2, b0, b0 source, MSE, [at ceiling] note
  per axis; skipped axes shown as N/A
- Move ESO block before report generation so results and PNG link
  are available when FlightReport is assembled
- ESO PNG link added to png_links on successful plot write

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

@nerdCopter Thanks for doing the thorough HEAD validation! All five remaining findings are confirmed resolved or no longer applicable:

  • --eso-axis removal eliminates the error-message inconsistency and any_valid guard findings entirely
  • eso_b0_flag_present properly guards duplicate --eso-b0
  • pctl_idx replaces the hardcoded p95_idx name
  • ESO_OMEGA0_MAX now has an explanatory comment

PR #136 is clean — no outstanding review issues remain. 🎉

(ノ◕ヮ◕)ノ*:・゚✧ 🐇

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant