Scientists, Stem Cells and Self-Delusion?
Alexandra Nowbar and colleagues at Imperial College London analyzed 133 reports from 49 trials using bone marrow stem cells in patients with heart disease. The effect evaluated was "mean left ventricular ejection fraction." They found 604 “discrepancies” in design, methods or results, based only on careful reading of the published reports. That is unfortunate, to say the least, but here’s the really disturbing finding:
The more discrepancies, the better the reported outcome.
The high-discrepancy group (5 trials with over 30 discrepancies each) showed a mean effect size of 7.7%. The next group (3 trials with 21–30) showed 5.7%. Those with 11–20, 3.0%. Those with 1–20, 2.1%. And:
[I]n the five trials without discrepancies the effect of bone marrow stem cell therapy on ejection fraction is zero.
Nature, in an editorial, calls this “a shocking reality check.”
There is more. Two papers by Harvard’s Piero Anversa and colleagues that described regeneration as a result of stem-cell treatment have been withdrawn or questioned. Another new study, published in Nature, suggests that regeneration may be at a “functionally insignificant level.” A survey published by The Cochrane Library did find some "moderate quality evidence” that bone marrow stem cells were beneficial, but then that’s what almost all the studies Nowbar et al. addressed found, too.
The analysis of stem cell studies clearly suggests confirmation bias — the tendency to search for or interpret information in a way that confirms one's preconceptions. That does not completely invalidate the research, but it certainly raises questions, both about methodology and about the peer review process.
There are, of course, other manifestations of bias. A group of scientists recently published in PLoS Biology a study of male bias, in what Nature News called “the case of the missing vaginas”:
Analyzing 25 years of research in the evolution of genitals, the authors found a strong bias towards studying male animals — a disparity that has got worse over time.
That clearly points to (presumably unconscious) bias.
Scientists are people, and these things will happen, so peer reviewers and other gatekeepers clearly need to be more aware of what can go wrong and catch such errors early. The stem cell field was terribly damaged by fraud a decade ago. It would be truly tragic if patients have had their hopes raised again only to be dashed.
Previously on Biopolitical Times: