Skip to content

[Repo Assist] feat: add Mann-Whitney U test (Wilcoxon rank-sum) for two independent samples#376

Draft
github-actions[bot] wants to merge 2 commits intodeveloperfrom
repo-assist/improve-mannwhitney-20260415-b912a34050ffbff2
Draft

[Repo Assist] feat: add Mann-Whitney U test (Wilcoxon rank-sum) for two independent samples#376
github-actions[bot] wants to merge 2 commits intodeveloperfrom
repo-assist/improve-mannwhitney-20260415-b912a34050ffbff2

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

🤖 This is an automated pull request from Repo Assist, an AI assistant.

Summary

Adds MannWhitneyTest.create — the Mann-Whitney U test (also known as the Wilcoxon rank-sum test) for comparing two independent samples. This complements the existing WilcoxonTest which handles paired (signed-rank) samples.

This addresses requests in issues #117 and #213 for a two-sample nonparametric test.

What's new

New file: src/FSharp.Stats/Testing/MannWhitney.fs

open FSharp.Stats.Testing

let sample1 = [| 6.; 7.; 8.; 9.; 10. |]
let sample2 = [| 1.; 2.; 3.; 4.; 5.  |]

let result = MannWhitneyTest.create sample1 sample2 true
// result.U1          = 25.0   (sample1 dominates all 25 pairs)
// result.U2          = 0.0
// result.Statistic   = z-score (normal approximation)
// result.PValueRight ≈ 0.006  (H1: sample1 > sample2)
// result.PValue      ≈ 0.012  (two-sided)

MannWhitneyTestStatistics record:

Field Description
U1 Times a value from sample 1 precedes a value from sample 2 (ties count as 0.5)
U2 Times a value from sample 2 precedes a value from sample 1; U1 + U2 = n1×n2
Statistic Z-score from normal approximation
PValueLeft P(sample 1 stochastically < sample 2)
PValueRight P(sample 1 stochastically > sample 2)
PValue Two-sided p-value

Implementation details

  • Ranks the combined sample using Rank.RankAverage() (average ranks for ties)
  • U1 = R1 − n1(n1+1)/2, where R1 is the rank sum of sample 1
  • Normal approximation: μ = n1n2/2, σ2 = (n1n2/(N(N−1))) × ((N3−N−Σtk(tk2−1))/12)
  • Optional continuity correction (±0.5) — recommended for small samples; matches R's correct=TRUE
  • Result values cross-checked against R's wilcox.test(..., exact=FALSE)

Test Status

✅ All 1201 tests pass (8 new tests added in Testing.fs):

Test Description
U statistics - clear separation U1=25, U2=0 for completely separated samples
U1 + U2 = n1*n2 Invariant holds for all three test cases
p-values - clear separation, no cc PValueRight < 0.01
p-values - clear separation, with cc PValue ≈ 0.0122 (matches R)
U statistics - moderate overlap U1=15, U2=10
p-value not significant - moderate overlap p > 0.05
U statistics with ties U1=4, U2=12 (tie correction applied)
p-values with ties PValueLeft ≈ 0.121, PValue ≈ 0.243

Trade-offs / limitations

  • Uses normal approximation only; exact p-values for small samples (N < 20) are not yet implemented. The function works but results may be less accurate for very small samples.
  • Does not currently expose n1 and n2 in the result record (can be inferred from input lengths).

Generated by 🌈 Repo Assist, see workflow run. Learn more.

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repo-assist.md@97143ac59cb3a13ef2a77581f929f06719c7402a

… samples

Implements MannWhitneyTest.create in Testing/MannWhitney.fs:
- Computes U1 and U2 statistics via rank sums
- Normal approximation with optional continuity correction
- Standard variance tie correction (all 3 counts handled)
- Returns MannWhitneyTestStatistics with U1, U2, Statistic (z), PValueLeft, PValueRight, PValue
- 8 new tests covering clear separation, moderate overlap, and tied data
- All 1201 tests pass

This complements the existing WilcoxonTest (signed-rank, paired samples)
with the rank-sum test for two independent samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@codecov-commenter
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.50000% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 47.92%. Comparing base (3533493) to head (5169712).
⚠️ Report is 185 commits behind head on developer.

Files with missing lines Patch % Lines
src/FSharp.Stats/Testing/MannWhitney.fs 75.86% 2 Missing and 5 partials ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##           developer     #376      +/-   ##
=============================================
+ Coverage      47.61%   47.92%   +0.30%     
=============================================
  Files            152      136      -16     
  Lines          16765    12967    -3798     
  Branches        2253     1717     -536     
=============================================
- Hits            7983     6214    -1769     
+ Misses          8097     6231    -1866     
+ Partials         685      522     -163     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant