Skip to content

feat: plot on numerical differences #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
WardDeb merged 1 commit into from
Oct 22, 2026
Merged

feat: plot on numerical differences #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions benchmark/benchmark.smk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ include: 'rules/bamcoverage.smk'
include: 'rules/bamcompare.smk'
include: 'rules/multibamsummary.smk'
include: 'rules/computematrix.smk'
include: 'rules/verify.smk'
include: 'rules/plotter.smk'
include: 'rules/benchmark_summary.smk'
include: 'rules/download_data.smk'
Expand Down Expand Up @@ -192,3 +193,4 @@ rule all:
'results/sample_stats.tsv',
'results/benchmark_summary.tsv',
'results/numerical_diff.tsv',
'results/numerical_diff_hist.png',
10 changes: 5 additions & 5 deletions benchmark/results/benchmark_summary.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
tool mean_speedup median_speedup mean_mem_dt3_gb mean_mem_dt4_gb mean_mem_diff_pct mean_mem_diff_gb
alignmentsieve 1.55 1.44 1.32 0.06 -94.9 -1.27
bamcompare 3.09 3.09 2.13 8.01 301.31 5.88
bamcoverage 4.64 4.94 1.26 3.53 158.45 2.27
computeMatrix 7.3 7.81 5.06 6.22 21.08 1.15
multibamsummary 5.65 5.34 2.44 0.47 -74.67 -1.98
alignmentsieve 1.43 1.32 1.32 0.06 -94.88 -1.26
bamcompare 3.06 3.06 2.12 8.0 301.5 5.87
bamcoverage 4.54 4.7 1.26 3.53 158.36 2.27
computeMatrix 7.94 8.65 5.06 6.33 21.65 1.28
multibamsummary 5.74 5.49 2.44 0.46 -74.64 -1.97
46 changes: 23 additions & 23 deletions benchmark/results/numerical_diff.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
tool sample n_reads_dt3 n_reads_dt4 n_common_reads_sampled n_pos_mismatch n_qlen_mismatch n_flag_mismatch n_values pearson_r max_abs_diff shape_dt3 shape_dt4 shape_mismatch n_bins_dt3 n_bins_dt4 n_common_bins n_common_bins_sampled
alignmentSieve alsieve_human_chip 29727000.0 29727000.0 20000.0 0.0 0.0 0.0
alignmentSieve alsieve_human_rna 97744386.0 97744386.0 20000.0 0.0 0.0 0.0
alignmentSieve alsieve_human_wgs 934945514.0 934945514.0 20000.0 0.0 0.0 0.0
alignmentSieve alsieve_triticum_chip 46427552.0 46427552.0 20000.0 0.0 0.0 0.0
alignmentSieve alsieve_triticum_rna 19232922.0 19232922.0 20000.0 0.0 0.0 0.0
alignmentSieve alsieve_triticum_wgs 1051310232.0 1051310232.0 20000.0 0.0 0.0 0.0
bamCoverage bcov_human_chip 3099750718.0 1.000000000800915 0.05078125
bamCoverage bcov_human_rna 3099750718.0 1.0000000000248048 4.75
bamCoverage bcov_human_wgs 3099750718.0 0.9999999851626744 0.05078125
bamCoverage bcov_triticum_chip 14547261565.0 0.9999999953596062 0.010009765625
bamCoverage bcov_triticum_rna 14547261565.0 1.0000000000500875 0.0625
bamCoverage bcov_triticum_wgs 14547261565.0 0.9999999674945368 0.0498046875
bamCompare bcom_human_chip 3099750718.0 0.9999956220626666 0.0050001144409179
bamCompare bcom_triticum_chip 14547261565.0 0.99999115944461 0.0050001144409179
multiBamSummary mbs_human_3 60000.0 1.0 0.0 (310096, 6) (310096, 6) False 310096.0 310096.0 310096.0 20000.0
multiBamSummary mbs_human_15 300000.0 0.9999999999999968 3.0 (310096, 18) (310096, 18) False 310096.0 310096.0 310096.0 20000.0
multiBamSummary mbs_triticum_3 60000.0 1.0 0.0 (1454737, 6) (1454737, 6) False 1454737.0 1454737.0 1454737.0 20000.0
multiBamSummary mbs_triticum_15 300000.0 0.9999999999999992 0.0600000000558793 (1454737, 18) (1454737, 18) False 1454737.0 1454737.0 1454737.0 20000.0
computeMatrix cm_human_3 12000000.0 1.0 0.0031250000029103 (200310, 600) (200310, 600) False
computeMatrix cm_human_15 60000000.0 0.9999999999999823 0.0031250000029103 (200310, 3000) (200310, 3000) False
computeMatrix cm_triticum_3 12000000.0 1.0 0.0007810000024619 (146597, 600) (146597, 600) False
computeMatrix cm_triticum_15 60000000.0 0.9999999999999856 0.0007810000024619 (146597, 3000) (146597, 3000) False
tool sample n_reads_dt3 n_reads_dt4 n_identical n_only_dt3 n_only_dt4 n_flag_mismatch n_qlen_mismatch n_cigar_mismatch n_mapq_mismatch chroms_only_dt3 chroms_only_dt4 n_values pearson_r n_nan_mismatch n_nonidentical frac_identical max_abs_diff max_abs_diff_loc max_rel_diff n_diff_gt_1e-06 n_diff_gt_0.001 n_diff_gt_0.1 shape_dt3 shape_dt4 shape_mismatch n_bins_dt3 n_bins_dt4 n_common_bins
alignmentSieve alsieve_human_chip 29727000.0 29727000.0 29727000.0 0.0 0.0 0.0 0.0 0.0 0.0
alignmentSieve alsieve_human_rna 250070850.0 250070850.0 250070850.0 0.0 0.0 0.0 0.0 0.0 0.0
alignmentSieve alsieve_human_wgs 934945514.0 934945514.0 934945514.0 0.0 0.0 0.0 0.0 0.0 0.0
alignmentSieve alsieve_triticum_chip 46427552.0 46427552.0 46427552.0 0.0 0.0 0.0 0.0 0.0 0.0
alignmentSieve alsieve_triticum_rna 22650946.0 22650946.0 22650946.0 0.0 0.0 0.0 0.0 0.0 0.0
alignmentSieve alsieve_triticum_wgs 1051310232.0 1051310232.0 1051310232.0 0.0 0.0 0.0 0.0 0.0 0.0
bamCoverage bcov_human_chip 3099750718.0 0.999999999001162 0.0 1835797727.0 0.4077595606834859 0.05078125 1:125179260 0.0014522272584559 1835797727.0 1352121305.0 0.0
bamCoverage bcov_human_rna 3099750718.0 0.9999999999991716 0.0 301140058.0 0.9028502336490144 4.75 14:49586730 0.0002870736248038 301140058.0 42751849.0 24690.0
bamCoverage bcov_human_wgs 3099750718.0 0.9999999894310986 0.0 2908998687.0 0.0615378617036261 0.05078125 3:93470430 0.0431545081092797 2908998687.0 2310496679.0 0.0
bamCoverage bcov_triticum_chip 14547261565.0 0.9999999922069556 0.0 2889008442.0 0.8014053415420251 0.010009765625 1B:80735790 0.0004098165701451 2889008442.0 1416822765.0 0.0
bamCoverage bcov_triticum_rna 14547261565.0 0.9999999999885468 0.0 154092080.0 0.9894074854355588 0.0625 3A:430336380 0.0010941214464004 154092080.0 97398130.0 0.0
bamCoverage bcov_triticum_wgs 14547261565.0 0.9999999675942488 0.0 13761865696.0 0.0539892587680986 0.0498046875 3B:101914160 0.0387326507523322 13761865696.0 10339414376.0 0.0
bamCompare bcom_human_chip 3099750718.0 0.9999956228802924 0.0 2909691427.0 0.0613143792164773 0.0050001144409179 1:2659940 1.0 2909691407.0 2313910977.0 0.0
bamCompare bcom_triticum_chip 14547261565.0 0.9999911613864244 0.0 13807666841.0 0.0508408211879154 0.0050001144409179 1A:722100 1.0 13807666711.0 12187591376.0 0.0
multiBamSummary mbs_human_3 930288.0 0.9999999999999984 0.0 6.0 0.9999935503843972 3.0 21:8210000-8220000 sample1 1.3432190674782218e-07 6.0 6.0 4.0 (310096, 6) (310096, 6) False 310096.0 310096.0 310096.0
multiBamSummary mbs_human_15 4651440.0 0.9999999999999982 0.0 30.0 0.9999935503843972 3.0 21:8210000-8220000 sample1 1.3432190674782218e-07 30.0 30.0 20.0 (310096, 18) (310096, 18) False 310096.0 310096.0 310096.0
multiBamSummary mbs_triticum_3 4364211.0 0.9999999999999996 0.0 4.0 0.999999083454031 0.0600000000558793 Un:93410000-93420000 sample2 7.382661280557464e-08 4.0 4.0 0.0 (1454737, 6) (1454737, 6) False 1454737.0 1454737.0 1454737.0
multiBamSummary mbs_triticum_15 21821055.0 0.9999999999999996 0.0 20.0 0.999999083454031 0.0600000000558793 Un:93410000-93420000 sample2 7.382661280557464e-08 20.0 20.0 0.0 (1454737, 18) (1454737, 18) False 1454737.0 1454737.0 1454737.0
computeMatrix cm_human_3 120185364.0 0.9999999999999996 0.0 1173160.0 0.9902387448774544 0.0500000000465661 row94221 col459 2.821192800318795e-05 847660.0 984.0 0.0 (200310, 600) (200310, 600) False
computeMatrix cm_human_15 600926820.0 0.9999999999999998 0.0 5865800.0 0.9902387448774544 0.0500000000465661 row94221 col459 2.821192800318795e-05 4238300.0 4920.0 0.0 (200310, 3000) (200310, 3000) False
computeMatrix cm_triticum_3 87957837.0 1.0 0.0 590633.0 0.9932850440603718 0.0007810000024619 row1809 col59 6.230529601420289e-08 547708.0 0.0 0.0 (146597, 600) (146597, 600) False
computeMatrix cm_triticum_15 439789185.0 0.9999999999999998 0.0 2953165.0 0.9932850440603718 0.0007810000024619 row1809 col59 6.230529601420289e-08 2738540.0 0.0 0.0 (146597, 3000) (146597, 3000) False
Loading
Loading