Ugh. This one is painful. In the comments to yesterday’s post about chiropractic, drlisamarie was kind enough to provide me with a link to an article (the abstract for the original paper can be found here) discussing the apparent benefits of chiropractic adjustment for sufferers of dyslexia and other learning disorders. While it appears this article was meant as an ‘in your face’ to yours truly, it is little more than another sad example of what passes for evidence in chiropractic. With apologies to Orac: The stupid. It burns.
The first step in evaluating this paper will be seeing where it was published and by whom. The study’s lead author is Yannick Pauli. Yannick runs a clinic specializing in chiropractic care of children suffering from ADHD, learning disabilities, and behavioral disorders. He treats these complex and multi-causal conditions by screwing with the spines of children. The study itself was published in the Journal of Vertebral Subluxation Research. A study’s merits are not and should not be defined by the journal it was published in. However, top tier journals are considered top tier because of their track record. Publications such as The New England Journal of Medicine and The Journal of the American Medical Association generally print well designed, scientifically valid studies. On the other end of the spectrum, there are journals with, shall we say, less rigorous standards. That brings us to the JVSR. The JVSR is a peer-reviewed journal published by and for that special brand of quack, the straight chiropractor. Briefly, straights believe that any and all ailments are the result of spinal subluxations (misalignment of the vertebrae) that block the flow of innate intelligence (the chiropractic version of life force) to the affected part of the body. A realignment of the displaced vertebral body will unblock the flow of life force and restore health to the patient (for a more thorough discussion of subluxation theory and why it’s bunk, visit the SBM article I linked yesterday). As you can see, we’re not off to a good start. However, just because the study was published in the JVSR and the lead author is a first-class quack doesn’t mean the study itself is not valid. Stranger things have happened (maybe). What about the study itself? Can it stand on its own? Well… no. Let’s explore why.
First, it’s important to note that the paper in question isn’t a report of a clinical trial but a systematic review of already published studies. Reviews are helpful tools in understanding the medical literature, but even when done well they have clear drawbacks. When not done well, they’re meaningless. The study linked by drlisamarie was not done well. With any review, it’s imperative that the authors establish clear cut guidelines of how papers will be included and excluded prior to the beginning of the search in an attempt to prevent selection bias. According to the abstract, the authors did set forth such guidelines. In a well-designed review, these parameters would exclude any trials that were too small, were not randomized, were not properly blinded, and/or were not placebo-controlled. Obviously, these cornerstones of scientific medicine were unimportant to Pauli and his colleagues. Four of the studies dealt with anecdotal evidence (you can’t make this stuff up), while the other four were before/after studies. Translation: no blinding, no controls, no randomization. If the studies being reviewed are worthless, the review will be worthless. That’s hardly a revelation. In addition, the review’s focus was much too broad to be significant. Systematic reviews must have a very fine focus because it is impossible to meaningfully combine studies that are looking at different populations, different outcomes, etc. However, that’s just what the authors did, as evidenced by the title of the paper (The Effects of Chiropractic Care on Individuals Suffering from Learning Disabilities and Dyslexia). By casting such a wide net, the authors invalidated their study before they even began the search that yielded those horrible papers mentioned above. It seems likely that the authors of this paper had already reached their conclusion before beginning their search, and then simply found evidence that validated their preconceived notions. Anyone familiar with the scientific method can tell you that this is the exact opposite of how it works. Welcome to alternative medicine.