Checks and balances in science

4 January 2013. Across the globe, there has been a recent surge of reported scientific fraud incidents in fields ranging from medicine and chemistry to social psychology. Some scientists produced fake reviews of their own or friends’ research papers. In other cases, scholarly papers were retracted because data had been fabricated. A Japanese anesthesiologist was found to have fabricated data in no less than 172 papers.

These fraud incidents come on top of high-profile plagiarism cases, such as those disclosed in Germany that involved two (now former) federal ministers. These authors simply copied and pasted other people’s texts into their own work. Compared to such old-school cases, the deception involved in recent scientific fraud incidents is far more sophisticated.

Manipulated data
A report published in late 2012 investigating fraud by a famous Dutch professor of social psychology who had fabricated or manipulated data in at least 34 scholarly publications serves as a case in point. He claimed to have conducted experiments at different institutions than where he was employed, without involving his colleagues or co-authors. There was no peer review of how the experiments were conducted, allowing him to make up or “massage” the data. He mixed verifiable details and fiction so that the findings seemed possible, thus lulling co-authors and reviewers into believing that the data collection process had been genuine. Finally, he often worked with less-experienced colleagues, whom he could intimidate.

How can we identify scientific fraud? There are parallels with the corporate governance scandals involving firms such as Enron, Swissair, Lehman or Mahindra Satyam. These companies had typically experienced aggressive international growth, and had been acclaimed by the press and the markets. Due to skewed incentive structures, they had poor internal control mechanisms, which led to fraudulent external reporting. Boards and other stakeholders typically ignored early warning signals.

In the world of science, as in business, we need adequately functioning checks and balances. Researchers, the journals that publish their results, their research communities and the institutions where they work all should contribute to preventing scientific fraud.

Getting our house in order
Preventing fraud is not rocket science. Here are some suggestions:

First, cheaters are typically individuals using primary data. Checks and balances should reduce the risk that individuals exploit the lack of transparency associated with collecting primary data. One way to increase reliability is to require qualified peers to oversee experiments or other forms of primary data collection. These peers should be respected academics identified by name, who are not co-authors of the same or another paper by the primary researcher.

Second, top journals should invest more in verifying research outcomes, e.g. by routinely sending external auditors to their authors. Nothing disciplines a researcher like knowing that his work will be audited in due course. Top journals could also require researchers to publish their data and the data collection process online for external use and evaluation.

Third, some research communities have started to develop a retraction index for journals. This gives an indication of the quality of a journal’s review and editorial process.

Fourth, research organisations should acknowledge that their skewed incentive structures have contributed to the recent fraud cases, just like excessive executive compensation has contributed to corporate governance scandals. Research organisations should move to a more balanced system that rewards excellence in teaching, research, knowledge transfer and self-governance alike.

In academia, we have just woken up to the issue of fraud. We’ve always known that the existing system based on peer review and journal reputation was not watertight. But recent affairs have shown that the scientific building increasingly looks like a leaky home. We should apply lessons learned in corporate governance to get our house back in order.

Picture: Photocase / Zettberlin