Skip to content

[MetaROR] Content gaps #28

Description

@LukasWallrich

Substantive content to add, mostly to balance reproduction vs replication coverage and to add hands-on guidance (R1, R2, R3).

  • Balance reproduction coverage — reproduction gets far less space than replication (essentially two sentences in Section 2). Move the Table 4.1 classification earlier and present reproduction subcategories (numerical vs robustness) more prominently. (R2)
  • Add parallel reproduction sections in Chapter 7 — currently missing "Defining Reproduction Success" and "Interpreting Reproduction Failures". (R3)
  • Expand data-integrity checking — even without raw data, partial checks are possible beyond statcheck/papercheck. This "reproduction light" approach deserves more than five lines; reference Heathers (2025) here too, not just for study selection. (R3)
  • Add hands-on practical guidance — address practical decisions like the "choice vs strategy" problem (should a replicator include the same covariates or apply the same selection rule?), and explain how to actually use tools like Altmetric rather than name-dropping them. (R3)
  • Add a discussion of data standardisation / FAIR principles — FAIR-ification addresses practical and scientific implications of data planning, acquisition and dissemination, and helps with human-in-the-loop issues (Section 5). (R2)
  • Add a discussion of incentives and career implications — current treatment (Koole & Lakens, 2012, in one table cell) is insufficient; expand on career risks and benefits of replication work, especially for ECRs. (R2)
  • Balance the MRI deep dive — Section 9 goes much deeper than other topics. Either pare it back to match the rest, or add comparable depth for another field; also add a section on identifying field-specific challenges for any discipline. (R1) (see depth-vs-breadth in Points for discussion)
  • Discuss what "the same" means in replication — the ambiguity of "doing the same" (same choices vs same decision strategy) is a central practical dilemma; address the "choice vs strategy" problem explicitly. (R3) (also flagged in Chapters 5 and 6)
  • Add a "checking adherence to preregistration" discussion — currently placed in reproduction (p. 27) but unclear how it fits there; move or justify. (R3)
  • Explore counter-narratives for mixed results — when the original paper reports a mix of significant and non-significant results across studies, discuss how to interpret this (integrous reporting? context dependence? same direction with varying significance vs opposing directions?). (R1)

Metadata

Metadata

Labels

metaror-reviewTasks from the MetaROR review todo

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions