Understanding Scientific Studies: Good vs. Bad Research Explained

Scientific discoveries often gain a certain momentum in the media. When it comes to topics that interest almost everyone - such as weight loss, anti-aging or life expectancy - we tend to accept study results without question. Especially when they offer a supposedly simple solution or a "miracle cure". But what is the real value of studies?

Put simply: A study can be anything, from a simple survey in the university cafeteria to a double-blind, placebo-controlled longitudinal study with thousands of participants. There are all sorts of different study designs, each with its own advantages and disadvantages. When the media reports on study results, they rarely go into detail - the conclusion has to fit into an attention-grabbing headline. We also don't always find out whether the study was carried out on mice or humans: the public often simply extrapolates the results from rodent studies to humans. This is sometimes misleading, but sometimes it is simply wrong.

Of mice and men – a word about rodent studies

Rodent studies are definitely justified. They are a first step in testing the safety and effectiveness of drugs or nutritional supplements. But that does not mean that all observations made on mice also apply to humans. Firstly, the metabolism of mice and humans works differently. What works for rodents does not necessarily work for us. Secondly, laboratory mice live in a "genetically controlled" environment: their strains differ much less than the genetic variations of humans. And finally, the lifestyle of the rodents also plays a role. They are kept in an artificial habitat - every factor in their environment can be influenced in a targeted manner. Of course, this is not the case for us humans either.

This is why there are sensational media reports on a regular basis that praise individual active ingredients as a cure-all. After all, there is a study on the subject! One example is the hype surrounding resveratrol, which was sparked by a crucial study: mice that were given high doses of resveratrol lived longer and healthier. Within a short time, dietary supplements containing the ingredient flooded the market. However, the results of the much-discussed study could not be reproduced in follow-up studies. This was due, among other things, to the mouse strain used, which was not representative of broader genetic populations - the transferability of the data to human users is therefore doubtful.

Reading and understanding scientific studies

Is every study published in a specialist journal also a relevant study? As a layperson, this is sometimes difficult to assess. We have put together a few aspects here that will help you evaluate scientific studies more easily. Roughly speaking, there are three types of studies: observational studies, experimental studies and analyses/reviews of these studies. You can see how you can classify the various study designs in a scientific context here.

So where do you find which information within a study? You can use the following points as a guide:

• Abstract or summary: This part gives a concise overview of the purpose, the methodology used and the results of the study.
• Methods: This is where the methodology is explained, i.e. how the study was conducted – this is a very important indication of its later significance.
• Results and discussion: In this section, the collected data are interpreted and discussed, and possible implications and limitations of the study are described.

You should also take a look at the funding of the study and any potential conflicts of interest. Manufacturer-funded studies, for example, do not always have a good reputation, because manufacturers naturally want to have scientific evidence of the effectiveness of their product. This is also an example of a conflict of interest: If researchers are funded by manufacturers, is their professional assessment still independent?

Another important quality feature for studies is the so-called peer review process: Before publication in a specialist journal, external researchers review the work of their colleagues. This is to ensure that the study meets scientific quality standards.

Conclusion

Evaluating and understanding studies is not always easy, even for experts - and there is a constant debate in the scientific community about methods and results. At ASPRIVA, we have decided to place great value on study quality. For example, we only consider human studies when we design products. And our scientific advisory board ensures that each study is carefully selected. You can find out more about our quality standards here.

Would you like to find out more? Then we recommend the following content:

• The Drive #269 - Good vs. bad science: how to read and understand scientific studies - Peter Attia
• Medical Research and the Myth of Scientific Truth | Stanford Humanities Center
• Dr. John Ioannidis Exposes the Bad Science of Colleagues - The Atlantic
• The Mouse Trap - How One Rodent Rules The Lab
• The Drive #143 - John Ioannidis, M.D., D.Sc.: Why most biomedical research is flawed, and how to improve it - Peter Attia
• The Drive #281 ‒ Longevity drugs, aging biomarkers, and updated findings from the Interventions Testing Program (ITP) | Rich Miller, M.D., Ph.D. - Peter Attia
• ITP | National Institute on Aging | National Institutes of Health
• There is a worrying amount of fraud in medical research | The Economist