Thursday, August 20, 2015

Bad Science, Fraud, and Peer Review - Updated

Updated below.

Article: A Scientific Look at Bad Science

Statistics: 

  • "By one estimate, from 2001 to 2010, the annual rate of retractions by academic journals increased by a factor of 11..."
  • "A study in the Proceedings of the National Academy of Sciences reviewed 2,047 retractions of biomedical and life-sciences articles and found that just 21.3 percent stemmed from straightforward error while 67.4 percent resulted from misconduct, including fraud or suspected fraud (43.4 percent) and plagiarism (9.8 percent)." 
  • In 2012, a researcher then at the biotechnology company Amgen wrote in Nature that when his team tried to reproduce 53 landmark cancer studies, they could replicate just six." [Ed: that is, about 11%]
  • "And according to a news report in Nature, a project aiming to reproduce the findings of 100 psychology papers has managed to replicate results for only 39 of them (the project’s findings are still under peer review)." 
A retracted paper means that it is has been published, after being peer reviewed. And 43% of the papers above were peer reviewed and later found to involve fraud.

What is peer review?
Peer review means that a board of scholarly reviewers in the subject area of the journal review materials they publish for quality of research and adherence to editorial standards of the journal before articles are accepted for publication. If you use materials from peer-reviewed publications they have been vetted by scholars in [the] field for quality and importance.  
Peer review is often waved as a magic wand conferring legitimacy on a controversial paper. While peer review may find some misconduct, it is mostly used for quality control.

The article points out that the need for a paper to be "important" is actually driving the misconduct. The research reported must have "positive" results, even though a lot research give "negative" results. However, negative results are important, but they are not "sexy" enough for the popular press, the scientific press and for the peer reviewers.

At its worst, peer review has been used to weed out results that are controversial and go against the "consensus" position in a field.

One reviewer of papers in neurobiology was notorious for demanding that the paper reference something about prions, and that the paper had to agree with the reviewer's position regarding prions.

Press Release: Retraction of articles from Springer journals
Springer confirms that 64 articles are being retracted from 10 Springer subscription journals, after editorial checks spotted fake email addresses, and subsequent internal investigations uncovered fabricated peer review reports.
And how many have not been found?

No comments:

Post a Comment