Skip to content

Add Unserved energy Conditional Value at Risk (CVAR) metric#100

Merged
abdelrahman-ayad merged 55 commits into
mainfrom
aa/CVAR
Jun 14, 2026
Merged

Add Unserved energy Conditional Value at Risk (CVAR) metric#100
abdelrahman-ayad merged 55 commits into
mainfrom
aa/CVAR

Conversation

@abdelrahman-ayad

@abdelrahman-ayad abdelrahman-ayad commented Feb 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR adds the Conditional Value at Risk (CVAR) metric to quantify tail-risk shortfalls in resource adequacy assessments. CVAR measures the expected unserved energy across the worst (1−α) simulation samples, where α is a user-defined confidence level (e.g., 0.95).

This PR implements CVAR for unserved energy on both ShortfallResult and ShortfallSamplesResult. CVAR for other dimensions (duration, magnitude) is out of scope and left for a follow-up.

Methodology

CVAR is computed as the weighted average of total unserved energy samples that exceed the α-th percentile (VAR). For ShortfallResult, an additional vector of per-sample total unserved energy is accumulated during simulation and discarded after CVAR computation (Type 1 in figure below).

implementation

alpha = 0.95
ue_cvar  = CVAR(:energy, shortfall, alpha) # system-level
ue_ncvar = NCVAR(shortfall, cvar) # normalized by demand
regional_ue_cvar = CVAR.(:energy, shortfall, alpha, sys.regions.names)
Screenshot 2026-02-23 at 12 52 37 PM

Performance testing

  • Tested the increase in memory and computation time using @benchmark testing for Shortfall and ShortfallSamples. There is a slight increase in memory use in the Shortfall due to the additional vector storing the total unserved energy across all samples.
Screenshot 2026-04-03 at 9 52 05 AM

@abdelrahman-ayad abdelrahman-ayad self-assigned this Feb 2, 2026
@abdelrahman-ayad abdelrahman-ayad added the enhancement New feature or request label Feb 2, 2026
@codecov-commenter

codecov-commenter commented Feb 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.80952% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.16%. Comparing base (fbde94e) to head (cc51916).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
PRASCore.jl/src/Results/ShortfallSamples.jl 72.22% 10 Missing ⚠️
PRASCore.jl/src/Results/Shortfall.jl 92.30% 2 Missing ⚠️
PRASCore.jl/src/Results/metrics.jl 90.00% 2 Missing ⚠️
PRASCore.jl/src/Results/utils.jl 88.23% 2 Missing ⚠️
PRASCore.jl/src/Results/Results.jl 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #100      +/-   ##
==========================================
+ Coverage   83.13%   83.16%   +0.02%     
==========================================
  Files          45       45              
  Lines        2325     2429     +104     
==========================================
+ Hits         1933     2020      +87     
- Misses        392      409      +17     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@abdelrahman-ayad abdelrahman-ayad marked this pull request as draft February 23, 2026 19:59
Agent-Logs-Url: https://github.com/NatLabRockies/PRAS/sessions/012d8c7b-67f5-45ad-82ee-c3db6a6e2379

Co-authored-by: abdelrahman-ayad <70848794+abdelrahman-ayad@users.noreply.github.com>

Copilot AI commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/JuliaArrays/FillArrays.jl/tarball/2f979084d1e13948a3352cf64a25df6bd3b4dca3
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/HDF5_jll.jl/tarball/e94f84da9af7ce9c6be049e9067e511e17ff89ec
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/Hwloc_jll.jl/tarball/baaaebd42ed9ee1bd9173cfd56910e55a8622ee1
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/Libiconv_jll.jl/tarball/be484f5c92fad0bd8acfef35fe017900b0b73809
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/MPICH_jll.jl/tarball/9341048b9f723f2ae2a72a5269ac2f15f80534dc
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/MPItrampoline_jll.jl/tarball/36c2d142e7d45fb98b5f83925213feb3292ca348
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/MicrosoftMPI_jll.jl/tarball/bc95bf4149bf535c09602e3acdf950d9b4376227
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/OpenMPI_jll.jl/tarball/2f3d05e419b6125ffe06e55784102e99325bdbe2
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/OpenSpecFun_jll.jl/tarball/1346c9208249809840c91b26703912dff463d335
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/Rmath_jll.jl/tarball/58cdd8fb2201a6267e1db87ff148dd6c1dbd8ad8
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/XML2_jll.jl/tarball/80d3930c6347cfce7ccf96bd3bafdf079d9c0390
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/Xorg_libpciaccess_jll.jl/tarball/4909eb8f1cbf6bd4b1c30dd18b2ead9019ef2fad
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/libaec_jll.jl/tarball/13b760f97c6e753b47df30cb438d4dc3b50df282
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaCollections/AbstractTrees.jl/tarball/2d9c9a55f9c93e8887ad391fbae72f8ef55e1177
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaCollections/DataStructures.jl/tarball/e86f4a2805f7f19bec5129bc9150c38208e5dc23
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaCollections/OrderedCollections.jl/tarball/05868e21324cede2207c6f0f466b4bfef6d5e7ee
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaCollections/SortingAlgorithms.jl/tarball/64d974c2e6fdf07f8155b5b2ca2ffa9069b608d9
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaData/DataAPI.jl/tarball/abe83f3a2f1b857aac70ef8b269080af17764bbe
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaData/Missings.jl/tarball/ec4f7fbeab05d7747bdf98eb74d130a2a2ed298d
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaData/Parsers.jl/tarball/7d2f8f21da5db6a806faf7b9b292296da42b2810
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaData/StructTypes.jl/tarball/159331b30e94d7b11379037feeb9b690950cace8
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaDocs/DocStringExtensions.jl/tarball/7442a5dfe1ebb773c29cc2962a8980f47221d76c
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaIO/HDF5.jl/tarball/e856eef26cf5bf2b0f95f8f4fc37553c72c8641c
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaLang/Compat.jl/tarball/9d8a54ce4b17aa5bdce0ea5c34bc5e7c340d16ad
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaLang/PrecompileTools.jl/tarball/07a921781cab75691315adc645096ed5e370cb77
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaMath/HypergeometricFunctions.jl/tarball/68c173f4f449de5b438ee67ed0c9c748dc31a2ec
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaMath/IrrationalConstants.jl/tarball/b2d91fe939cae05960e760110b328288867b5758
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaMath/QuadGK.jl/tarball/9da16da70037ba9d701192e27befedefb91ec284
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaMath/SpecialFunctions.jl/tarball/2700b235561b0335d5bef7097a111dc513b8655e
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaPackaging/JLLWrappers.jl/tarball/0533e564aae234aff59ab625543145446d8b6ec2
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaPackaging/Preferences.jl/tarball/8b770b60760d4451834fe79dd483e318eee709c4
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaPackaging/Requires.jl/tarball/62389eeff14780bfe55195b7204c0d8738436d64
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaPackaging/Scratch.jl/tarball/9b81b8393e50b7d4e6d0a9f14e192294d3b7c109
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaParallel/MPI.jl/tarball/8e98d5d80b87403c311fd51e8455d4546ba7a5f8
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaPlots/Plots.jl/tarball/5c3d09cc4f31f5fc6af001c250bf1278733100ff
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaRandom/Random123.jl/tarball/dbe5fd0b334694e905cb9fda73cd8554333c46e2
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaRandom/RandomNumbers.jl/tarball/c6ec94d2aaba1ab2ff983052cf6a606ca5985902
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaStats/Distributions.jl/tarball/fbcc7610f6d8348428f722ecbe0e6cfe22e672c6
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaStats/LogExpFunctions.jl/tarball/13ca9e2586b89836fd20cccf56e57e2b9ae7f38f
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaStats/PDMats.jl/tarball/e4cff168707d441cd6bf3ff7e4832bdf34278e4a
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaStats/Rmath.jl/tarball/5b3d50eb374cea306873b371d3f8d3915a018f0b
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaStats/Statistics.jl/tarball/ae3bb1eb3bba077cd276bc5cfc337cc65c3075c0
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaStats/StatsAPI.jl/tarball/178ed29fd5b2a2cfc3bd31c13375ae925623ff36
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaStats/StatsBase.jl/tarball/aceda6f4e598d331548e04cc6b2124a6148138e3
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaStats/StatsFuns.jl/tarball/91f091a8716a6bb38417a6e6f274602a19aaa685
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaStrings/InlineStrings.jl/tarball/8f3d257792a522b4601c24a577954b0a8cd7334d
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaTesting/ExprTools.jl/tarball/27415f162e6028e81c72b82ef756bf321213b6ec
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaTesting/Mocking.jl/tarball/2c140d60d7cb82badf06d8783800d0bcd1a7daa2
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaTime/TZJData.jl/tarball/72df96b3a595b7aab1e101eb07d2a435963a97e2
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/JuliaTime/TimeZones.jl/tarball/d422301b2a1e294e3e4214061e44f338cafe18a2
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/LilithHafner/AliasTables.jl/tarball/9876e1e164b144ca45e9e3198d0b689cadfed9ff
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/LilithHafner/PtrArrays.jl/tarball/4fbbafbc6251b883f4d2705356f3641f3652a7fe
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/NREL/MinCostFlows.jl/tarball/8f52000f0db84f9bda8114c874410e69ce36f067
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/NREL/PRAS/tarball/097bf724e73aeb09710c8fc67180a5981248bc9e
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/NREL/PRAS/tarball/24d15e10fa4676c67b78c0ecc663ef56969c13a1
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/NREL/PRAS/tarball/8f0742091618b908b73ac38be8800b064e8d07b1
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/joshday/OnlineStats.jl/tarball/a19ba00b3968afcde81471033fd22976e62f5fea
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/joshday/OnlineStatsBase.jl/tarball/a5a5a68d079ce531b0220e99789e0c1c8c5ed215
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/quinnj/JSON3.jl/tarball/411eccfe8aba0814ffa0fdf4860913ed09c34975
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • https://api.github.com/repos/simonster/Reexport.jl/tarball/45e428421666073eab6f2da5c9d310d99bb12f9b
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (http block)
  • pkg.julialang.org
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.test(&#34;PRAS&#34;) (dns block)
    • Triggering command: /usr/bin/julia julia --project=PRAS.jl -e using Pkg; Pkg.develop(path=&#34;PRASCore.jl&#34;) Pkg.instantiate() -ljulia -ljulia-internal (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Comment thread PRASCore.jl/src/Results/metrics.jl
Comment thread PRASCore.jl/src/Results/Shortfall.jl Outdated
Comment thread PRASCore.jl/src/Results/Shortfall.jl Outdated
Comment thread PRASCore.jl/src/Results/Shortfall.jl Outdated
Comment thread PRAS.jl/test/runtests.jl Outdated
Comment thread PRASCore.jl/src/Results/Shortfall.jl Outdated
Comment thread PRASCore.jl/src/Results/Shortfall.jl Outdated
Comment thread PRASCore.jl/src/Results/metrics.jl Outdated
Comment thread PRASCore.jl/src/Results/ShortfallSamples.jl Outdated
Comment thread PRASCore.jl/src/Results/Results.jl Outdated
Comment thread PRASCore.jl/src/Results/metrics.jl Outdated
Comment thread PRASCore.jl/src/Results/metrics.jl Outdated
Comment thread PRASCore.jl/src/Results/Results.jl Outdated
Comment thread PRASCore.jl/src/Results/Results.jl Outdated

@sriharisundar sriharisundar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address any explicit comments I have here (moving _cvar and _ncvar to utils.jl, potentially rename dim to quantity, print CVAR unit based on dim rather than just using E), and merge!

@sriharisundar

Copy link
Copy Markdown
Member

LGTM, please squash and merge. We will update versions across packages when we merge couple other PRs. Thanks for the patience :)

@abdelrahman-ayad

Copy link
Copy Markdown
Collaborator Author

Thanks Hari, will merge now.

@abdelrahman-ayad abdelrahman-ayad merged commit c793303 into main Jun 14, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants