I saw a couple of things this weekend that illustrated how the research process can be essentially corrupted by combinations of perverse incentives and moral hazard. In the FT, a lot of Anjana Ahuja’s review of a book on such problems in the bio-medical sector rang very true to my experience of real estate research – and I’m pretty sure that it applies to most other disciplines where the stakes may be lower. There’s a lot of reference in the piece to the reproducibility crisis in research. If you replace the words ‘patients’ with ‘participants’ and ‘animals’ with ‘students’ (sorry) in the piece below, then you could basically say the same thing about lots of academic research.
But the reproducibility crisis also attests to the many ways in which scientists can sway or manipulate their results. These techniques include, but are not confined to, HARKing (hypothesising after the results are known), creative statistical analyses, choosing particular patient subsets, having too few people in a study, and over-extrapolating from animal studies.
There was another piece in the Guardian by Will Storr looking at the growth of the self-esteem “business”, grade inflation and forms of self-absorption. Behind it all is a fascinating example – but I suspect not an unusual one – of how commissioners can influence research projects to get the results that they want to hear and how researchers will sometimes respond to that pressure in order to secure financial support.
Ben Goldacre has been campaigning passionately for years about the problems of publication bias in medical research – but I suspect that its’s endemic across most disciplines. It’s something that I’ve written about before but it’s definitely worth a reminder.