Making research evaluation more transparent: Aligning research philosophy, departmental values, and reporting

Abstract

There is a growing interest in changing the culture of psychology to improve the quality of our science. At the root of this interest is concern over the reproducibility of key findings. A variety of large-scale replication attempts have revealed that several previously published effects cannot be reproduced, whereas other analyses indicate that the published literature is rife with underpowered studies and publication bias. These revelations suggest that it is time to change how psychological science is carried out and increase the transparency of reporting. We argue that change will be slow until institutions adopt new procedures for evaluating scholarly activity. We consider three actions that individuals and departments can take to facilitate change throughout psychological science: the development of individualized research-philosophy statements, the creation of an annotated curriculum vitae to improve the transparency of scholarly reporting, and the use of a formal evaluative system that explicitly captures behaviors that support reproducibility. Our recommendations build on proposals for open science by enabling researchers to have a voice in articulating (and contextualizing) how they would like their work to be evaluated and by providing a mechanism for more detailed and transparent reporting of scholarly activities.

James A. Grand

Associate Professor, Psychology

A scientist at heart, an organizational psychologist by training, and a lucky dad and husband all the time.