MSstats shiny vs MSstats in R discrepancies

[martinsindelar0077-ui]

Dear MSstats developers,
I am experiencing systematic discrepancies between results exported from the MSstats Shiny app and results reproduced using the exported R script, despite using identical settings and the same input data. Looking at exported group comparison for some proteins there is a huge difference in MissingPercentage parameter exported directly from Shiny interface vs using R script - e.g. from 0.0187 to 0.2939 for the same exact protein which affects also other values like FC. Is MSstats shiny doing something different than MSstats in R?

Thank you,
Martin

[tonywu1999]

@martinsindelar0077-ui

Thank you for reporting this problem. This is definitely unexpected behavior.

Would you be able to share a sample of your dataset where you're experiencing this discrepancy and then we can debug the problem on our end?

[martinsindelar0077-ui]

Unfortunately, I can’t share my dataset due to confidentiality, but I can share the R script. If that’s not sufficient, I can try to reproduce my results using some publicly available dataset. Here, when exploring the data in the ‘model’, discrepancies are already appearing.

`

Load Packages

library(MSstats)
library(MSstatsTMT)
library(MSstatsPTM)

setwd("C:/Martin/Proteomics/MSstats/test_data")

Package versions

MSstats version 4.10.1

MSstatsTMT version 2.10.0

MSstatsPTM version 2.4.4

Read data

data = read.csv("report_test.tsv", header = TRUE, sep = '\t')
annot_file = read.csv("MSstats_annotation.csv")#Optional
data = DIANNtoMSstatsFormat(data,
annotation = annot_file, #Optional
qvalue_cutoff = 0.01, ## same as default
removeProtein_with1Feature = TRUE,
use_log_file = FALSE)
data = unique(as.data.frame(data))

use MSstats for protein summarization

summarized = MSstats::dataProcess(data,
normalization = 'equalizeMedians',
logTrans = 2,
nameStandards = c(''),
featureSubset = 'all',
n_top_feature = NULL,
summaryMethod="TMP",
censoredInt='NA',
MBimpute=TRUE,
remove50missing=FALSE,
maxQuantileforCensored=0.999)
dataProcessPlots(data=summarized,
type="ProfilePlot",
ylimUp = FALSE,
ylimDown = FALSE,
which.Protein = "ADHE_ECOLI",
summaryPlot = TRUE,
address = FALSE,isPlotly=TRUE)

Create the contrast matrix

contrast.matrix = NULL
comparison = matrix(c(-1, -1, -1, 1, 1, 1, 0, 0, 0, 0),nrow=1)
contrast.matrix = rbind(contrast.matrix, comparison)
row.names(contrast.matrix)=c("41hvs0h")
colnames(contrast.matrix)=c("2F_0h_WCO1","2F_0h_WCO2","2F_0h_WCO3","2F_41h_WCO1","2F_41h_WCO2","2F_41h_WCO3","3F_0h_WCO3","3S_0h_WCO4","3S_41h_WCO4","3S_64h_WCO4")

Model-based comparison

model = MSstats::groupComparison(contrast.matrix, summarized)
groupComparisonPlots(data=model$ComparisonResult,
type="VolcanoPlot",
which.Comparison="41hvs0h",
which.Protein="ADHE_ECOLI",isPlotly=FALSE,
address="")`

[martinsindelar0077-ui]

in MSstats shiny all options for pre-processing were checked
Use unique peptides
Remove proteins with 1 peptide and charge
Remove proteins with 1 feature
Filter with Q-value (0.01)

for everything else default settings were used
data used - report from DIANN

[martinsindelar0077-ui]

It seems to me that Shiny does not apply the 'Use unique peptides' option even when it’s checked. When I removed this line from my script:
data = unique(as.data.frame(data))
I got the same results as the Shiny GUI.
I also tested analyzing the data with and without checking the option in MSstats Shiny, and both gave me identical results. However, when I run the analysis directly in R, I get different results depending on whether I remove that line or keep it.

Or is the Use unique peptides option doing something different than data = unique(as.data.frame(data))?
In any case, the exported script always includes this line (even if Use unique peptides is not checked), and with that line included, it produces different results.

[tonywu1999]

@swaraj-neu

I did a preliminary look. Here's what I found:

In the main_calculations.R file, there's a line:

prep_input = MSstatsHandleMissing(prep_input, qc_input$summaryMethod, qc_input$MBi, ..)

qc_input$censInt is missing it seems. That may be related.

[swaraj-neu]

Hi @martinsindelar0077-ui ,

Thanks again for the detailed report and especially for narrowing the
discrepancy down to the unique() line. That was the key clue.

The fixes are now merged into the devel branch of MSstatsShiny:

1. The unconditional data = unique(as.data.frame(data)) line has been
   removed from the exported R script generator. This was the main
   cause of the MissingPercentage / FC discrepancy you reported.
2. Separately, the "Assumptions for missing values" (censoredInt) radio
   button was being silently ignored in the UI's internal pipeline.
   That is now passed through correctly.

Would you be able to install the devel version and rerun your analysis
to confirm the discrepancy is resolved on your dataset?

devtools::install_github("Vitek-Lab/MSstatsShiny", ref = "devel")

One honest note: in our internal testing on a public DIA-NN dataset,
the UI and the exported R script now agree very closely on
MissingPercentage and downstream values, but a small residual difference
(a few percent on some metrics) can remain. This is due to a separate
architectural detail in how the UI orchestrates preprocessing vs how
MSstats::dataProcess() does, which we're tracking as a follow-up. If
the numbers you see still look off in a way that affects your results
in practice, please let us know and we'll keep digging.

Thanks again for the report.

CC: @tonywu1999