diff --git a/discussion.qmd b/discussion.qmd index 619070d..84268fa 100644 --- a/discussion.qmd +++ b/discussion.qmd @@ -172,6 +172,27 @@ Whether that generalises to the setting of the original study needs to be considered in light of theory, and might be a legitimate matter of contention. +These difficulties point to a partly irreducible source of disagreement +about replication success. Because experimental control is limited and +the space of possible moderators is vast, experts often cannot decide +unambiguously whether divergent results reflect a genuine failure to +replicate the original effect or a violation of the *ceteris paribus* +assumption between the original study and the replication, a version of +Collins' experimenter's +regress that Sarisoy [-@Sarisoy2025] examines in detail. On this account, +sustained disagreement about whether a replication succeeded can reflect +legitimate normative judgements that researchers make when the evidence +underdetermines the conclusion, rather than necessarily indicating poor research practice. +Sarisoy argues that such disagreements become more tractable once +researchers are transparent about a replication's intended *epistemic +function*, whether it is designed to test the reliability (stability) of +an effect, to probe a specific validity threat, or to assess +generalisation to a new context, because each function carries different +standards for what would count as success. Declaring this purpose, in +addition to pre-specifying which effects are of primary interest (see +@sec-success-criteria), helps to recast debates as disagreements about +what a replication was meant to show. + ## The Role of Differences for the Interpretation of Findings {#sec-differences-and-interpretation} Each replication outcome should be evaluated in the light of its diff --git a/references.bib b/references.bib index cecd4e5..4265670 100644 --- a/references.bib +++ b/references.bib @@ -1359,6 +1359,17 @@ @article{RosenbergFinn2022 doi = {10.1038/s41593-022-01110-9} } +@article{Sarisoy2025, + author = {Sarisoy, J.}, + title = {Why we disagree about the success of replications}, + journal = {Journal for General Philosophy of Science}, + volume = {56}, + number = {3}, + pages = {307-324}, + year = {2025}, + doi = {10.1007/s10838-024-09709-1} +} + @article{SchauerHedges2021, author = {Schauer, J. M. and Hedges, L. V.}, title = {Reconsidering statistical methods for assessing replication},