Skip to content

feat: Support loading environment variables from a file#178

Merged
mraszyk merged 5 commits into
dfinity:mainfrom
dsarlis:dimitris/env-variables
May 18, 2026
Merged

feat: Support loading environment variables from a file#178
mraszyk merged 5 commits into
dfinity:mainfrom
dsarlis:dimitris/env-variables

Conversation

@dsarlis
Copy link
Copy Markdown
Contributor

@dsarlis dsarlis commented May 13, 2026

Add support to load environment variables from a file.

Some canisters might expect that certain environment variables are set when calling certain endpoints, so with this change we make it easier to have such canisters be benchmarked without the need to disable any environment variable related logic and write benchmarks with minimal code instrumentation as usual.

The idea is very similar to how stable memory can be instantiated from a file, essentially exposing another option in canbench.yml to support environment variables in a similar fashion. The format of the file is CSV which feels natural for a list of key, value pairs like environment variables.

@dsarlis dsarlis requested a review from a team as a code owner May 13, 2026 13:16
Comment thread canbench-bin/Cargo.toml
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 13, 2026

canbench 🏋 (dir: ./examples/btreemap_vs_hashmap) f7cc166 2026-05-15 07:35:07 UTC

./examples/btreemap_vs_hashmap/canbench_results.yml is up to date
📦 canbench_results_btreemap-vs-hashmap-example.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 13, 2026

canbench 🏋 (dir: ./examples/fibonacci) f7cc166 2026-05-15 07:35:11 UTC

./examples/fibonacci/canbench_results.yml is up to date
📦 canbench_results_fibonacci-example.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 5 | regressed 0 | improved 0 | new 0 | unchanged 5]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 5 | regressed 0 | improved 0 | new 0 | unchanged 5]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 5 | regressed 0 | improved 0 | new 0 | unchanged 5]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@mraszyk mraszyk requested a review from Copilot May 13, 2026 14:27
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support in canbench for preloading canister environment variables from a user-provided file, enabling benchmarking of canisters that depend on ic_cdk::api::env_var_value(...) without additional instrumentation.

Changes:

  • Extends canbench.yml parsing to accept an env_vars.file path and threads it through benchmark execution.
  • Creates canisters in PocketIC with CanisterSettings.environment_variables populated from the file.
  • Adds new test canisters + CLI tests covering success, missing file, and invalid file cases, plus updates crate docs.

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/environment_variables/src/main.rs New test canister that reads env vars at init and asserts expected values in a benchmark.
tests/environment_variables/environment_variables.csv Sample env var data file used by the test canister.
tests/environment_variables/canbench.yml Test config exercising env_vars.file.
tests/environment_variables_invalid/src/main.rs Minimal canister for invalid env var file test.
tests/environment_variables_invalid/environment_variables_invalid.csv Invalid env var file fixture (no comma separator).
tests/environment_variables_invalid/canbench.yml Test config pointing at invalid env var file.
tests/environment_variables_file_does_not_exist/src/main.rs Minimal canister for missing env var file test.
tests/environment_variables_file_does_not_exist/canbench.yml Test config pointing at a non-existent env var file.
tests/Cargo.toml Registers the new test canister binaries.
Cargo.lock Locks new dependency additions.
canbench-rs/src/lib.rs Documents environment-variable file support in the crate-level docs.
canbench-bin/tests/tests.rs Adds integration tests for env var loading / error output.
canbench-bin/src/main.rs Adds env_vars to config parsing and passes the path into run_benchmarks.
canbench-bin/src/lib.rs Implements env var file parsing and applies it via CanisterSettings during canister creation.
canbench-bin/Cargo.toml Adds ic-management-canister-types dependency for EnvironmentVariable.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread canbench-rs/src/lib.rs Outdated
Comment thread canbench-bin/src/lib.rs Outdated
Comment thread canbench-bin/src/lib.rs Outdated
Comment thread tests/environment_variables/src/main.rs Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment thread canbench-rs/src/lib.rs Outdated
Comment thread canbench-bin/tests/tests.rs Outdated
Comment thread canbench-rs/src/lib.rs Outdated
mraszyk
mraszyk previously approved these changes May 13, 2026
Co-authored-by: mraszyk <31483726+mraszyk@users.noreply.github.com>
@github-actions github-actions Bot dismissed mraszyk’s stale review May 13, 2026 17:36

Review dismissed by automation script.

dsarlis and others added 2 commits May 13, 2026 20:38
Co-authored-by: mraszyk <31483726+mraszyk@users.noreply.github.com>
@dsarlis dsarlis requested a review from mraszyk May 13, 2026 17:45
@dsarlis
Copy link
Copy Markdown
Contributor Author

dsarlis commented May 18, 2026

@mraszyk Feel free to merge when ready.

@mraszyk mraszyk merged commit a126fc7 into dfinity:main May 18, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants