Sometimes I feel like a lot of us feel overwhelmed, bewildered, or just plain turned off by the debates around EDs, Evidence Based Medicine, different therapies & treatments, and all the research and data we throw around this joint. In a good way! Even if sometimes our passions make the discussions contentious, *that* the discussions are happening is very valuable, IMHO. But I think they could be even *more* valuable.
What I'd like to do on this thread is give readers a practical tutorial with the following goals:
- Explain the basics of medical research;
- Explain the relationship between research and the practice of medicine;
- Give readers the tools to critique Mainstream Media (MSM) reporting of primary research;
- Give readers the tools to evaluate a primary research paper for their own understanding.
The truth is that no research is perfect, some research is poorly designed or executed, and no treatment for any disease or condition is uniformly effective. We think of Medicine as a Science. It is NOT, it is an Art that is *informed* by science. And actually, it's a GOOD thing that Medicine does not rely 100% on science - it would be bad if a discipline as complicated and encompassing as Medicine had to wait for and rely on only one source of information. That said, the science behind Medicine is REALLY IMPORTANT!
Step 1: Understanding Medical Research -Different Kinds of Research Give You Different Kinds of Information.
The first thing is to understand that the "knowledge base" of medicine comes from many sources. Each of these has strengths and weaknesses. The FIRST thing to do when you are looking at a piece of research is to identify which category it falls under. Then you have an idea of what kind of information the study design is good at getting, and what the limitations are. This is by no means exhaustive, but it's a good starting point :
- Primary bench science: this would be the kind of experiments NOT run on people at all, but more like the stuff most of us did in biology or chemistry in school. This would include experiments on animals.
- Pros: Valuable for understanding basic mechanisms, lots of control over conditions, you can easily repeat what was done.
- Cons: hard to know how well what happens in a lab will match what happens in a real live human being, lab conditions do not replicate real world conditions very well.
- Pros: Pretty much the only way to study rare or singular events, good for descriptive purposes.
- Cons: very small numbers, not repeatable, hard to know how generalizable (how applicable to another person or situation) the results or findings are.
- Pros: Inexpensive to do, easier to get large numbers, the study is already looking at "real life" situations, low risks to study participants.
- Cons: Can't control all of the conditions, can only show relationships but CANNOT show which thing causes what, can be hard to replicate.
- Pros: You can control conditions and participants so you don't have contamination from unknown sources. You can "prove" causality. Easy to duplicate
- Cons: Expensive, study participation limits can limit generalizability.
BUT! Not every problem can be investigated with an RCT. And you DON"T need an RCT if the preponderance of evidence reaches a certain tipping point. Real life example:
- Smoking has never been "proven" to cause lung cancer in humans. This is because it would be unethical to design a study where you make one group smoke. The preponderance of the evidence from primary bench science, animal studies, and observational human studies is that smoking *does* cause lung cancer. This did not stop scientists in the employ of tobacco companies from loudly proclaiming that the connection wasn't "proven" for decades. They were technically right, but ethically reprehensible.
The other issue is that RCTs participants are often very unlike the larger group of people who would get the treatment, and that the treatment is administered under ideal, and not real world conditions. Because RCTs can be very small in terms of numbers of participants, it can be hard to get detailed data about subgroups of patients - men vs. women, responders vs. not responders, that kind of thing.
So, take home of part 1: When reading a report about research, look for what kind of study it was. Think about the plusses and minuses of that kind of study. Then look at the results being reported and see what kind of "yeah but" (or caveat) might apply.
For further interesting comment, please read the italics that Carrie has added on her blog and/or follow the thread on the forum