Questionnaire | page 1

Welcome to the FORRT Replication Database Submission Form!

If you have conducted a replication of any social scientific study, you can enter your results here. We will validate the results via the sources that you enter. If everything is fine, they will be added to the FORRT Replication Database, if not, we will contact you through the e-mail address that you provide. In return for a complete and correct entry, we offer a contributorship (CRediT: investigation) to the person who entered the data: You will be listed on the FReD-website as a contributor and we will invite you to be a co-author on an eventual dataset publication (not the ReD JOPD Paper of a previous version and data from psychology but one in an interdisciplinary journal in 2025 or later). Please note that the data you enter will be public.

If you have any questions or need help filling in the form, check out a video tutorial walking you through the form or write us an e-mail (see the bottom of the page). The same goes for any sort of feedback! Data and code are available via the OSF project. If you want to enter multiple results from the same study at once, we recommend you consider doing this via the Google Spreadsheet. You can more information about how to contribute to FReD in this presentation.

Mandatory Entries

The following characteristics of your replication study are mandatory. Without them, we will not be able to add the study to the Replication Database.

Before entering your study, we kindly ask you to check if it is already included in the Replication Database. If this is the case for a study of which you are a contributor and you want to be listed as an author, please do not fill in this form but simply send us an e-mail.

I have checked the replication database at https://metaanalyses.shinyapps.io/replicationdatabase/ and searched all entries in the “Replicability Tracker” tab. The study that I want to submit is not yet listed there.

Please indicate if this is the only effect that you are submitting for this sample.

If there are multiple replication effects that correspond to the same sample, please enter a short reference below. For the other entries, you have to fill out this form again and repeat the exact same reference that you enter here. If there are many effects for you replication, you can also contact us to enter them in a different format.

To elaborate, with the same sample we refer to whether these are the same participants that will be used for additional replication database entries. This is the case if there are multiple items for which you report the effect sizes. This is not the case if you report separate effect sizes for an MTURK and a PROLIFIC sample (or male vs. female, student vs. non-student, etc.). In this case, please respond that this is the only effect for this sample.

Example references for multiple entries per sample:
- Smith & Meyer, 2020, Study 1

This is the only effect that I want to report for this sample.

I want to report multiple effects for this sample. For all these same samples, the ID is as follows:

What are the references of the original and replication study?

Please use APA recommendations for the references. In case of multiple studies, hypotheses, or tests, please also provide the details. Formatting (e.g., italics) does not matter in this case. Your replication study does not need to be published but the original study should be. If there is no corresponding journal article, poster, or pre-print for your replication study, please upload all necessary materials and data to an osf.io, researchbox.org, or any other repository so that you can provide a link and citation for your study. Please also provide DOIs for the respective references. If you have conducted the replication study yourself, it is not yet published with a DOI, you can create a DOI via an OSF project to which you upload the study materials.

Example citation for a journal article:
- Roelofs, A. (2008). Tracing Attention and the Activation Flow in Spoken Word Planning Using Eye Movements. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34, 353-368.
dx.doi.org/10.1037/0278-7393.34.2.353, Study 1

Example citation for a pre-print:
- Wolf, D., Röseler, L., Leder, J., & Schütz, A. (2022, June 15). The Red-Anger Effect: Is it Nothing More Than Demand Characteristics?. doi.org/10.31234/osf.io/ntukz, Study 1

Example citation for an OSF-project:
- Wolf, D., Röseler, L., & Schütz, A. (2022, September 13). Conceptual Replication (Young et al., 2016, Study 1). Retrieved from osf.io/7ezvb, Study 1

Example DOIs:
- 10.1016/j.jesp.2009.05.003
- 10.1037/a0019689

Original study reference
Replication study reference
Original study doi
Replication study doi

Please provide a description of the hypothesis or phenomenon that was investigated in the original and replication study.

If you report a single effect for your study, examples are “Orienting attention in visual working memory reduces interference from memory probes” or “heat priming”. If you plan to report multiple effects from the same sample or study, please specify your entry by adding in parentheses the item or sample for which your effect is reported.

For guidance please also see existing entries (docs.google.com/spreadsheets/d/1x68oW2H_Xrdv44fIeycl4fegsmQgCa60GxeZZ_hAR90/edit?pli=1#gid=1463805480), effects, and disciplines (forrt.org/reversals).

Description
Name of the effect
Name of the discipline
Is there nesting in the analyses? If so, please describe (e.g., 2x3 within-between interaction, pupils nested in schools, repeated measures on factor X, ...)
Name of the journal that published the original finding
Claim (paraphrased authors’ conclusion or interpretation of the evidence)
Statistical support (paraphrased statistical test results that support a key finding from the study
Pagenumber of the claim in the original article
Pagenumber of the statistical support in the original article

What are the results of the original and replication study?

You have the option to enter the effect size (r, d, etasquare, partial etasquare, cohens f, ...) or the test statistics. If you enter effect sizes, they will be converted to r if possible. If you report test statistics, r will be computed on the basis of them. Please note that effect size computation is more accurate if you provide the effect sizes from your study. If you provide effect sizes and test statistics, the latter will be ignored for the conversion but be visible in the dataset.

For effect size types please use the following abbreviations:
- Odds Ratio: or
- Cohen’s d: d
- (Partial) Eta Squared: etasq
- Correlation: r
- Cohen’s f: f
- R²: r²
- Cohen’s dz: dz
- All others: please provide sufficient detail (e.g., rather write Cramer’s V instead of “V”)

Effect size (original study)
example: 1.234

Effect size (replication study)
example: 1.234

Effect size type (original study)
example: d

Effect size type (replication study)
example: d

Sample size (original study)
example: 100

Sample size (replication study)
example: 100

Test statistic (original study)
example: t(143) = 0.35, p = .725 (two-tailed), dz = 0.029

Test statistic (replication study)
example: t(143) = 0.35, p = .725 (two-tailed), dz = 0.029

Reported p-value (original study; including ">" or "=" signs and tails if mentioned))

Reported p-value (replication study; including ">" or "=" signs and tails if mentioned)

Did you use the same design (e.g., within-subjects / between-subjects) and statistical test (e.g., t-test / AxB interaction in an AxBxC repeated Measures ANOVA) as the authors of the original study?

Please check those that apply.

same design

same statistical test

Please enter location where researchers can verify the values that you entered.

For effect sizes / test statistics / sample sizes / etc. please type in the reference and page number. If you are submitting results from an unpublished replication study, you can for example type in the link to a publically uploaded R-script and dataset with which your values can be reproduced.

Optional Entries

The following characteristics of your replication study are optional. Entering them will allow your study to be included in more complex analyses, such as comparisons between fields, and allow you and other researchers to explore moderators of replicability.

Please enter an e-mail address so that we can contact you in case there are any questions about your entry (optional).
Do you want to be listed as a contributor on the FReD website? If so, please enter your name and e-mail address below.

To see what this would look like, you can go to forrt.org/apps/fred_explorer.html and select the tab “About”.

Name (Last, First):

E-mail adress:

Please include me as a contributor.

How would you rate the outcome of the replication?

- success: The original effect could be replicated / Original and replication results converge / Replication results could confirm the original hypothesis

- informative failure to replicate: The replication study worked (i.e., planned sample size was achieved) as planned but could not confirm the original results.

- practical failure to replicate: There was a problem when conducting the replication study (e.g., manipulation did not work, not enough participants could be recruited, measures did not work, technical errors occured, ...). Thus, the replication results not converging with the original results can easily be attributed to the quality of the replication study.

- inconclusive: Some of the results could confirm the original hypothesis and some could not.

success

informative failure to replicate

practical failure to replicate

inconclusive

Please provide the link to a repository (e.g., osf.io, researchbox.org) where researchers can find your materials and dataset.

Link to study materials and results:

Was the replication study preregistered?

A pre-registration usually contains details about the hypotheses, methods, and analysis plan prior to the data being collected or analyzed. It is a document that is available online and cannot be edited. If your study was pre-registered, please enter the link below. If there is an embargo on the preregistration, type [embargoed until dd.mm.yyyy] behind it (e.g., “embargoed until 31.12.2024”).

Pre-registration link:

In which way has the replication study been published?

Unpublished

Pre-print

Journal Article

Other (thesis, data publication, OSF-project, etc.)

Was any of the original study’s authors involved in designing the replication?

yes, one or more author(s) of the original study have been involved in the replication study

no, none of the original authors have been involved in the replication study

Please check all that apply to indicate the closeness between the replication study and the original study.

Exact

Different

Does not Apply

Please specify all differences between the original study and the replication.

Differences between the original study and the replication:

Please describe the design of your study.

All observations are independent (e.g., cross-sectional study, between-subjects experiment)

Observations are not independent (e.g., multilevel structure, clustered sample, repeated measures)

Other, please describe:

To enhance findability for the entered replication, you can provide up to five tags below.

1
2
3
4
5

Are there any other comments or notes that you would like to make about this entry? If so, please enter them here.

By clicking "submit data", your data will currently not be directly streamed into the Replication Database. If you have provided an e-mail address, we will contact you once we have validated your entry. If you do not hear from us within two weeks, please write us an e-mail.