Check the facts

I always told my students, particularly bio students, that while their goal might have been to get a specific grade, get accepted into the program of their choice, etc, my goal for them was that they could evaluate claims in the news and understand what their doctor was saying. One would hope that peer reviewed (other experts in the field review articles to make sure that they are based on sound science) scientific articles are able to be taken at their word, but science is very rarely set in stone. Researchers might authentically find something, that no one else can find after them. Did they lie, not necessarily, but they hadn’t yet found the whole answer. Check out this article by Oransky and Marcus (2017) in STAT about how often science that makes it into popular news is actually overturned, and how infrequently those changes are reported.

A perhaps bigger problem is an increasing proliferation of journals that just aren’t valid. Spotty peer-review processes and pay-to-publish policies are just two ways that some journals are creating doubt in the process and the results. In one such journal, I found such a horrendous example of shoddy science that I had to share. Unfortunately, I have also found this article referenced on other sites that have used it as support for their claims.

Atik, Atik, and Karatepe (2016) supposedly studied the effects of applying apple cider vinegar as a topical rub on varicose vein symptoms, pain, and subsequent appearance anxiety. They say that it was a randomized (participants were assigned randomly) controlled (experimental group was compared to a control group) trial. They might have achieved their first claim of randomization. However, in a controlled study you need to have a control group to compare the treatment group to. Two things are important in setting up these groups – participants should not know which group they are in. This means the control group is generally given a placebo (a treatment that has no known therapeutic effect) that takes the same form as the actual treatment. The second thing is that only one variable should be different between the two groups – in the case of this study that means that both groups should have received and done the doctor recommended treatment, and the experimental group should have also applied apple cider vinegar (while the control group applied a placebo). Neither of these were done.

“Patients in the study group were suggested to apply apple vinegar to the area of the leg with varicosity alongside the treatment suggested by the doctor…The patients in the control group received no intervention during the study” (p. 2-3). That means there are two variables that are different between the two groups, and the control group knows that they aren’t doing anything, while the treatment group knows that they are, so you can’t separate psychological effects from actual treatment effects. Surprise, the experimental group improved more than the control group (and yes the control group still improved), but you can’t tell whether that was because of the doctor recommended treatment or the apple cider vinegar. So surely the researchers recognized that in the discussion and conclusion? Nope “we determined that the external application of apple vinegar on varicosity patients, which is a very easy application, increased the positive effects of conservative treatment” (p. 6). How!? There is nothing in the study on which to base this claim. How did this get through a peer review process?

This is sadly just one example of why individuals need to think critically. That doesn’t mean becoming a google doctor, but it does mean paying attention to the claims that are being made and checking the facts regardless of whether they support what you already know or believe.