Friday, May 17, 2019 - A new report released by the National Academies of Sciences, Engineering, and Medicine is weighing in on a contentious debate within the science world: the idea that scientific research is fundamentally flawed, rife with published findings that often can’t be reproduced or replicated by other scientists, otherwise known as the replication and reproducibility crisis.

Their report, the collaborative work of more than a dozen experts from universities and the private research world, doesn’t go so far as to call it a crisis. But it does call for wide-scale improvements in how scientists do their work, and it also takes scientists—and journalists—to task for sometimes overhyping the latest research findings.

For years now, some scientists have sounded a clarion call about the overall quality of published research. Common issues highlighted by these scientists have included fraudulent, poorly done, or overhyped studies, with embellished findings based on small sample sizes; statistical manipulation of a study’s results during or after the experiment is over to achieve a desired outcome; and studies with negative conclusions being suppressed by their authors or rejected by scientific journals, which can then skew the medical literature on a particular topic, such as a drug’s effectiveness.

The most glaring fallout from these issues has been that many of the most influential or flashy findings in science, particularly in psychology, fail to be reproduced (meaning that other researchers can’t get the same findings by crunching the same raw data obtained by the original study) or replicated (meaning that other researchers, when recreating the design of the original study, don’t get similar results).

The report also notes that the American public’s confidence in science hasn’t wavered at all in recent years, despite major news articles discussing the “crisis” in psychology and elsewhere. And it found that even scientists who have criticized the current state of things aren’t completely on-board with calling science broken.

The committee, while calling for a streamlined approach to replication and reproducibility studies as well as the better storage and availability of datasets for these studies to happen, also concludes that scientists shouldn’t worry too much about replicating any one individual study.

“A predominant focus on the replicability of individual studies is an inefficient way to assure the reliability of scientific knowledge,” they wrote. “Rather, reviews of cumulative evidence on a subject, to assess both the overall effect size and generalizability, is often a more useful way to gain confidence in the state of scientific knowledge.”

Of course, it isn’t just research methods that could stand to improve. The report also singles out journalists, citing a survey showing that 73 percent of Americans agree that the “biggest problem with news about scientific research findings is the way news reporters cover it.”

The report recommended that journalists “should report on scientific results with as much context and nuance as the medium allows,” especially when the research is complicated, contrary to what most similar studies have found on the same topic, or when the researchers involved have possible conflicts of interest, such as prior or current industry funding.

That presumably includes telling readers when a study involves mice instead of people.

Learn More at this Link