Thursday, May 2, 2019

Is scientific research well done?

Tabby Cat
Oliver-Bonjoch, CC BY-SA 3.0

Sometimes, while reading items published by journals such as Science News, it looks like some research currently being carried out is platitudinous. Either it leads to the discovery of things everyone knows, or time and efforts are spent to investigate in fields that no one cares about. We know that many researchers are anxious to publish, and they must justify somewhat the funds they receive, but up to that point?
Let’s look at a very recent news (April 2019):
Cats recognize their name. A study suggests our feline friends can tell the familiar sound of their name from other words. A paragraph of this news adds: As for whether or not a cat understands what a name is, well, only the cat knows that.
Anyone who has had a cat (I had one half a century ago) knows that cats recognize their name. Was it necessary to do a research about this, probably spending public money, to discover something that everyone knows?
Another contentious field in current scientific research is the confirmation of results. One of the fundamental rules of the scientific method is the fact that every discovery must be confirmed by a different team, perhaps by a different procedure (this is not always possible), before the discovery can be considered part of the stock of science. In recent times, apparently researchers don’t want to perform this kind of work, the confirmation of experiments carried out by others, probably because it does not lead to scientific recognition in the form of publications and projects. This means that many “discoveries” are parked without confirmation, or even worse: it is assumed that all those discoveries are genuine without having been confirmed, which transgresses one of the main principles of the scientific method. Let us look at a few recent signs of concern about this:
  • Psychology results evaporate upon further review. This August 2015 report analyzed 97 results published in important journals of the field. Just 35 of them could be reproduced. It is true that a few months later (April 2016) another analysis came to the conclusion that the previous pessimistic study was a false alarm, and that 85 of the analyzed works could really be replicated, but this new analysis is as controversial as the former, as many experts do not accept its optimistic conclusions.
  • Medicine is another field where these problems usually arise. For instance, certain animal studies concluded that erythropoietin (also known as Epo) could help cancer patients undergoing radiation and chemotherapy to overcome the anemia caused by the treatment. When the tests with human patients began, some doctors verified that the mortality of patients treated with Epo seemed to increase, so they didn’t dare to proceed with them. Other researchers who tried to reproduce the first favorable experiments were not successful. Seven years later (January 2015) we still don’t know whether Epo treatment is good or bad for cancer patients. The just quoted article explicitly says the following:
Unprecedented funding challenges have put scientists under extreme pressure to publish quickly and often. Those pressures may lead researchers to publish results before proper vetting or to keep hush about experiments that didn’t pan out.
The same article contains the following worrying data:
The Bayer pharmaceutical company tried to repeat studies in three research fields, mostly cancer studies. Almost two-thirds of the redos produced results inconsistent with the original findings.
  • In relation to a book by Stephen Stigler, published in 2016 and titled The Seven Pillars of Statistical Wisdom, Science News says that in many realms of science today, “statistical wisdom” seems to be in short supply. Misuse of statistics in scientific research has contributed substantially to the widespread “reproducibility crisis” afflicting many fields. 
It looks like we are going through a crisis in scientific research, at least in the fields of medical and social sciences. In another post I talked about a different crisis affecting physics, the most scientific of all fields. Where is science going?


The same post in Spanish
Thematic Thread on Science in General: Previous Next
Manuel Alfonseca

No comments:

Post a Comment