Avoiding Questionable Research Practices in Applied Psychology

The replication crisis in psychology and allied fields has exposed critical flaws in the standard views of research methods, which allow for extensive flexibility in data analysis by investigators and permit the widespread use of questionable research practices (QRPs). This authoritative volume presents a detailed analysis of the replication crisis and recommended practices for combatting these problems, including open data practices, preregistration of hypotheses and analyses, and adversarial collaborations. It will be a practical, useful resource for students and instructors in graduate and advanced undergraduate level research methods classes, as well as psychological researchers interested in improving their own research.

Format: Paperback / softback
Length: 447 pages
Publication date: 25 August 2023
Publisher: Springer International Publishing AG

The replication crisis in psychology and allied fields has exposed critical flaws in the standard views of research methods, which allow for extensive flexibility in data analysis by investigators and permit the widespread use of questionable research practices (QRPs). Chapters examine the intentional use of QRPs such as data fabrication and falsification, as well as subtler, unintentional practices such as p-hacking and HARKING (hypothesizing after results are known). Drawing on the growing awareness of these problems, contributors also highlight potential strategies to detect QRPs and minimize their negative impact through open data practices, preregistration of hypotheses and analyses, and adversarial collaborations, in which investigators holding opposing positions on a scientific issue agree to work together on a study in an effort to counteract their respective biases.

The replication crisis in psychology and allied fields has highlighted significant flaws in the traditional approaches to research methods, leading to extensive flexibility in data analysis and the widespread use of questionable research practices (QRPs). This authoritative volume presents a detailed analysis of the replication crisis and the use of QRPs, as well as recommended practices for combating these problems. Ultimately, the book aims to provide a comprehensive, current, and accessible account of the adverse effects of QRPs.

The replication crisis has revealed that QRPs can significantly undermine the reliability and trustworthiness of findings in psychology and the social sciences. Chapters in the book examine the intentional use of QRPs such as data fabrication and falsification, as well as subtler, unintentional practices such as p-hacking and HARKING (hypothesizing after results are known). Drawing on the growing awareness of these problems, contributors also highlight potential strategies to detect QRPs and minimize their negative impact through open data practices, preregistration of hypotheses and analyses, and adversarial collaborations, in which investigators holding opposing positions on a scientific issue agree to work together on a study in an effort to counteract their respective biases.

The book covers a wide range of topics, including the history of controversies in statistics and replication, embracing intellectual humility while designing research, confirmatory vs. exploratory analyses, publication bias and negative results, and promoting honest and transparent report writing. It provides a practical, useful resource for students and instructors in graduate and advanced undergraduate level research methods classes.

One of the key challenges facing psychology and allied fields is the intentional use of QRPs. Data fabrication and falsification are among the most egregious QRPs, involving the creation or manipulation of data to support a particular hypothesis or conclusion. These practices can be difficult to detect, as they often involve complex statistical techniques and data manipulation. However, there are several strategies that can be used to detect QRPs, including statistical analysis, peer review, and independent replication.

P-hacking is another QRP that involves manipulating data or analysis to obtain statistically significant results. This practice can be particularly problematic in fields such as psychology, where small sample sizes are common. P-hacking can be detected by using statistical techniques such as permutation testing, which randomly shuffles the data and compares the results to the original analysis.

HARKING is a practice that involves hypothesizing after results are known. This practice can be particularly problematic in fields such as psychology, where researchers may be tempted to interpret their results in a way that supports their hypothesis. HARKING can be detected by using statistical techniques such as meta-analysis, which combines the results of multiple studies to obtain a more accurate estimate of the effect size.

In addition to intentional QRPs, there are also subtler, unintentional practices that can undermine the reliability and trustworthiness of findings. These practices include selective reporting, selective analysis, and selective interpretation. Selective reporting involves reporting only the results that support a particular hypothesis or conclusion, while selective analysis involves analyzing only a subset of the data. Selective interpretation involves interpreting the results in a way that supports a particular hypothesis or conclusion.

To address these QRPs, researchers can take several steps. One of the most important steps is to embrace intellectual humility while designing research. This involves recognizing that there are many unknowns in the field and that it is important to approach research with a critical eye. Researchers should also be transparent about their methods and data, and should be willing to share their data and code with other researchers.

Preregistration of hypotheses and analyses is another important step in addressing QRPs. This involves specifying the hypotheses and analyses that will be conducted before the data is collected. This can help to prevent p-hacking and HARKING, as researchers will be required to follow a predetermined plan of analysis.

Adversarial collaborations are also a potential strategy for addressing QRPs. In these collaborations, investigators holding opposing positions on a scientific issue agree to work together on a study in an effort to counteract their respective biases. This can help to ensure that the research is conducted in a fair and unbiased manner.

In conclusion, the replication crisis in psychology and allied fields has exposed critical flaws in the standard views of research methods, which allow for extensive flexibility in data analysis by investigators and permit the widespread use of questionable research practices (QRPs). This authoritative volume presents a detailed analysis of the replication crisis and the use of QRPs, as well as recommended practices for combating these problems. By embracing intellectual humility while designing research, preregistering hypotheses and analyses, and engaging in adversarial collaborations, researchers can help to ensure that the research is conducted in a fair and unbiased manner. This will help to improve the reliability and trustworthiness of findings in psychology and the social sciences, and will benefit students and instructors in graduate and advanced undergraduate level research methods classes.

Dimension: 235 x 155 (mm)
ISBN-13: 9783031049705
Edition number: 1st ed. 2022