Extend discussion of disagreement about replication success (#4)

discussion.qmd

@@ -172,6 +172,27 @@ Whether that generalises to the setting of the original study needs to
 be considered in light of theory, and might be a legitimate matter of
 contention.
 
+These difficulties point to a partly irreducible source of disagreement
+about replication success. Because experimental control is limited and
+the space of possible moderators is vast, experts often cannot decide
+unambiguously whether divergent results reflect a genuine failure to
+replicate the original effect or a violation of the *ceteris paribus*
+assumption between the original study and the replication, a version of
+Collins' experimenter's
+regress that Sarisoy [-@Sarisoy2025] examines in detail. On this account,
+sustained disagreement about whether a replication succeeded can reflect
+legitimate normative judgements that researchers make when the evidence
+underdetermines the conclusion, rather than necessarily indicating poor research practice.
+Sarisoy argues that such disagreements become more tractable once
+researchers are transparent about a replication's intended *epistemic
+function*, whether it is designed to test the reliability (stability) of
+an effect, to probe a specific validity threat, or to assess
+generalisation to a new context, because each function carries different
+standards for what would count as success. Declaring this purpose, in
+addition to pre-specifying which effects are of primary interest (see
+@sec-success-criteria), helps to recast debates as disagreements about
+what a replication was meant to show.
+
 ## The Role of Differences for the Interpretation of Findings {#sec-differences-and-interpretation}
 
 Each replication outcome should be evaluated in the light of its

references.bib

@@ -1359,6 +1359,17 @@ @article{RosenbergFinn2022
   doi = {10.1038/s41593-022-01110-9}
 }
 
+@article{Sarisoy2025,
+  author = {Sarisoy, J.},
+  title = {Why we disagree about the success of replications},
+  journal = {Journal for General Philosophy of Science},
+  volume = {56},
+  number = {3},
+  pages = {307-324},
+  year = {2025},
+  doi = {10.1007/s10838-024-09709-1}
+}
+
 @article{SchauerHedges2021,
   author = {Schauer, J. M. and Hedges, L. V.},
   title = {Reconsidering statistical methods for assessing replication},