Abstract: Researchers often make analytic decisions while already seeing how the results are turning out. Although this is a natural part of exploratory work, it can also introduce bias into confirmatory analyses—for example, when analysts (consciously or not) favor analytic choices that yield statistically significant results. Such practices, often discussed under the label of p-hacking and other questionable research practices, have become a central concern in efforts to improve the credibility of scientific findings.

One proposed solution is analysis blinding, a methodological approach originally developed in astrophysics to prevent researchers from being influenced by preliminary results. Early discussions of the approach were introduced to behavioral scientists by researchers who argued that temporarily hiding hypothesis-relevant information from analysts could help reduce bias in statistical decision-making. In an analysis-blinding workflow, researchers develop, debug, and finalize their analysis pipeline using a modified dataset in which the critical relationships or labels are concealed. The true results are revealed only after analytic decisions have been finalized.

This workshop introduces a practical analysis-blinding workflow in R using the vazul package (https://nthun.github.io/vazul/). The package provides tools for creating blinded datasets through masking and scrambling procedures that preserve key distributional properties while concealing hypothesis-relevant relationships or labels. Participants will learn how to generate blinded datasets, develop and validate analysis scripts using these datasets, and safely unblind the data once the analytic decisions are finalized. The workshop aims to equip researchers with a concrete and easily adoptable workflow that reduces bias while preserving analytical flexibility.

Instructor: Dr. Tamás Nagy, Assistant Professor at ELTE Eötvös Loránd University (Budapest, Hungary) (ORCID)

Required skills: being able to use R for data analysis

Recommended skills: some prior experience with literate programming (R Markdown or Quarto) and version control (git and GitHub), however small, is advantageous

The workshop is supported by the Open Science Ambassador program of the Berlin University Alliance (BUA) and participation is free. Register here