Much of the emphasis of this class is toward increasing your understanding of what it means to have an evidence base for what you do as a chiropractor. Eventually we hope to understand how chiropractic adjustments work, which adjustments work best for what type of subluxation, even what are the best methods for determining when a patient needs our care. Despite the personal experiences you may have had and the confidence you may feel or have felt from chiropractors you know, the scientific evidence for a lot of what we do is still lacking.

Do you already believe that chiropractic works? How much evidence would it take to convince you that it does? (Zero, right?) On the other hand, how much evidence would it take to convince a skeptic that chiropractic works? (About the same amount that it would take to convince a believer that it doesn’t!)

Recall the article by Joe Keating on faulty logic used in the profession (see week 3 reading assignments). Here is an example of a true skeptic who is willing to accept only the highest quality of evidence. If we as a profession want to be accepted on the merits of our science, we need to aspire to that same level of evidence. There are certain rules that must be followed and certain methods that need to be used to guarantee that we accept as fact only those things that can be proven. Logic is a large part of it.

It’s not that immaterial or nonmechanistic processes don’t exist or aren’t important. It’s just that we can’t use our personal beliefs about those things as evidence to convince anyone else of their existence. This kind of approach should also help you distinguish marketing and sales type presentations from research presentations. At any point in reading or hearing a research presentation you should be able to ask "How does he or she know that?" There should be some kind of evidence that goes beyond personal bias toward or against a product or method.

As Keating pointed out, the most acceptable and believable information will be found in peer-reviewed research journals. The article on peer review and indexing (week 4 reading assignment) goes into some detail on the peer-review process and how it helps increase the validity of information. Still, as part of your training in research methods, you need to learn how to critically evaluate what you read, even in peer reviewed journals.

Critical reading is a bit of a skill. It’s based on a certain core of knowledge about research design, the ability to logically apply that knowledge, and an almost continual critical viewpoint. Keep asking yourself that question: "How do they know that?" In this article I’ll review the major types of articles you’ll see when you begin reading the literature and describe the levels of evidence you’ll see in the experimental methods. I hope you can use this knowledge as part of that core you’ll need to develop critical reading and thinking skills.

The articles you find in your literature search and as you browse the contents of journals are of several different types. This is most clearly seen if you take a look at a recent hard copy of a journal like JMPT. The table of contents is divided into sections: Original Articles, Case Reports, Commentary and Letters to the Editor. All these article types are considered peer-reviewed and will show up in your literature searches.

Commentaries and Letters-to-the-Editor are not really research articles, though they often contain interesting information. When anyone publishes an article in a journal, the readership has the opportunity to critique the article. While peer-review takes care of many problems, errors or misinterpretations can still creep in. The readers can submit their critique as a letter, which will be published in a subsequent issue of the journal. The original author of the article gets to respond, usually in that same issue. (That’s why, in Pubmed, you might see several citations with the same title, but different authors.) Sometimes arguments can span several issues. That’s a tip-off that the issue is contentious and still under investigation. Some journals also solicit commentary articles from prominent researchers. Commentaries are often reviews of a certain topic.

Case-reports and original articles are the ones that contain actual data and usually represent new research. Case reports are descriptions of the results of care of a single patient. They are interesting to read, but aren’t a very strong form of evidence. We’ll talk more about the limitations of case studies in the section below on design types. Other types of original articles will be related to the type of research design they employ, some will be experimental and some observational (terms you’ll see defined below). Keep a special look-out, though, for review articles.

A review article is a type of original research, but it’s based on an analysis of other published research. If you want to find out the general consensus of what’s known on a certain topic, seek out a review of that topic. (Some literature search indexes will let you select "Review" as an option, and will only return review articles.) In a review, the author will do her or his own literature search on a fairly focused topic and tell you what was found in that review, hence, the reference section will be enormous. Often the author will try to come up with a conclusion that balances the pro and con information. The most recent the review article is, the better, since it will contain the most recent source information.

The best reviews are called "systematic critical reviews." The "system" that makes it systematic is the process they use to find all the articles that deal with the topic at hand, whether it’s manipulative care for low back pain, or the reliability of subluxation measures. A good, honest, systematic review will not be biased in their selection of references. The "critical" part of the review is the way the authors judge the quality of the information found in the references. There’s usually a scoring method, including such things as whether the study was blinded, or the adequacy of the sample size, that allows the authors to assign a point score to rate the quality of the study. Information from weaker sources may be excluded from the conclusions, or just carry less weight in the analysis. A great source of systematic critical reviews is the Cochrane Collaboration library (www.cochrane.org).

So, when you’re beginning your own review of the literature for your research project, keep in mind that some of the work may have already been done for you. If you can find a good recent review of the topic, then you could use that as a starting point. (But not an ending point!)

The other original research articles you see in a journal probably use some kind of method to collect data and report on the findings. There are a few different research designs that might be used, and each has its strengths and weaknesses. In the end we hope to find out what kind of care will provide the best results in the patients chiropractors see in their offices. At the same time, we hope that the findings will be honest, accurate and free of bias. So research designs have to balance scientific factors with practicality to be useful. A few terms are used to describe studies in general:

The strongest research designs will be prospective and experimental and have ways to control for bias. Consider this table of research designs:

The basic design types are presented here in increasing order of quality. Anecdotes are those stories you always hear about a doctor getting great results in practice. The stories may be true, but they are often undocumented and there are no records.

The case report is a legitimate research design but it suffers from a couple of serious limitations. First, case reports are almost always retrospective, that is, chosen for publication after the fact. So, the tendency is to select only those cases where spectacular results were obtained. Who wants to hear about negative results? Hence, case reports almost always suffer from a "selection bias." They don’t show what typically happens in practice, they more often show unusual occurrences. Secondly, a case report is an observational study in that there is no control group. You cannot tell from a single case whether it was the care that produced the results or some other unobserved influence (more on non-specific effects later).

A series of cases is more convincing than a single case, because at least there is repeatability to the phenomenon. Still, without a control group, there is no way to prove that the care (adjustment) was the cause of the change.

There is an experimental, prospective type of case study, called the Time Series study (the 4^th one on the list above). In the time series design, the doctor makes a few baseline measures on the patient before care even begins (Kind of like the pattern analysis!). This baseline period becomes the control for the care period. (The patient is his or her own control.) If the condition or function measured is fairly stable during the baseline period, then abruptly changes after the onset of care, there is evidence that the care brought about the change. A slideshow with several examples of single-subject time-series articles is linked to your syllabus. (See the "slideshow" link in week 4).

A cross-sectional study is one in which a large number of patients are looked at on one occasion – a cross-section in time. Often these studies are done to describe the effects of care on patients in practitioners’ offices. You could call this type of study "Practice-Based." In a cross-sectional study you can record many variables on the patients, such as age, gender, original complaint, general health level, and length of time under chiropractic care. Statistical analysis would be used to draw some conclusions about relationships between those variables. For instance, Rupert used this design to show that elderly patients who had been under maintenance care for the longest periods of time were more healthy than patients of the same age that were under medical care only. (Do a Pubmed search on Rupert R AND chiropractic, it’ll pop right up.)

The retrospective case control and cohort studies are similar in that they look at the outcomes of care in a large number of patients. The retrospective study is weaker simply because there’s not as much chance to get data on all patients, if something was missed, you cannot go back and get it in a retrospective study. The cohort study is stronger; you can show what percentage of patients that undergo chiropractic care get good results. It’s somewhat like a case series, but on large numbers of patients, and it’s done prospectively, so there’s better control of data collection. Still, you cannot easily account for other influences that might have brought about the changes in health observed.

The randomized clinical trial (RCT) is the king of all research designs, often called "the gold standard." This is the classic design that involves two or more groups, one of which is a control group, hence it is experimental. Blinding of group assignment is desirable – you must blind the assessors at least, but you can attempt to blind the patient and care givers as well. The so called "triple blind" study blinds the assessor, the patient and the care-giver. The double blind only blinds two of the participants, usually the assessors and the patient. The best RCT’s also incorporate some kind of random assignment of patients to the groups. Randomization does two things; it first eliminates the bias on the part of the study coordinator who might send more severe patients into the chiropractic group. Secondly, a random assortment is more likely to put matching patients into different groups. An RCT would not be valid if the patients in the control group had much different characteristics (age, gender, severity of complaint) than patients in the treatment group.

Another issue in RCTs is the choice of "control" or sham care. The whole reason for the control group is to show that the particular drug or adjustment was the cause of any changes seen in the treatment group. You need the control group to experience all the same events and aspects of care as the treatment group with the exception of the drug or adjustment. A blinded control group can show whether changes that occur might have been due simply to contact with the physician or the placebo effect. For that reason, RCTs often try to incorporate a sham adjustment - something that looks like an adjustment and can fool the patient into thinking they have been adjusted. The complete classic design then would be called a "Randomized Double Blind Sham-Controlled Clinical Trial."

There’s still some debate about whether a true sham is even possible in chiropractic studies. You can almost never blind the doctor to whether they are giving a real adjustment or a sham, and it’s pretty hard to blind the patients, too. Also, the sham adjustment really needs to be non-effective. If the sham has a chance of correcting some subluxations, then you will get improvement in the control group as well as the adjustment group and the adjustment will appear less effective than it might really be.

It’s more common these days to have a comparison RCT where chiropractic care is compared to some other type of care for the same condition, such as medical care or physical therapy. There is no true sham control group, so you analyze the data to look for the relative benefit of one care system over the other.

Ultimately, it would be nice to demonstrate the hypothesis that chiropractic care is of benefit to patients suffering from a particular complaint. Or at least in what proportion of patients chiropractic care is effective. To satisfy the rules of evidence, you have to not only show that patients who got the care improved, you also need to show that the care was responsible for the change, not some other "non-specific" cause. Consider the following table:

On the left is a list of possible causes for improvements in patients’ health. Across the top is a list of research design types. From what you’ve read above, try and put checks into the boxes to show which design types can account for which effects.

Finally, consider the topic of generalizability. Again, you would like to be able to make use of research information in your own practice, with your own patients. Of the research designs above, the ones that are most like your practice setting are the practice-based research designs: case studies, case series, cross-sectional, and cohort. Those also test the effects of care on real patients like yours. Unfortunately, those are also the weakest designs and least likely to make a big splash in the literature or in high profile reviews.

On the other hand, the strongest design, the RCT, is performed almost always in some kind of research institution setting, nothing like a practice setting. They also tend to use only a special class of patients, complaining of only the one condition under investigation -- again, nothing like the regular patients you might see in practice. The care is usually some single technique often excluding the other forms of care or education you might use as adjuncts. Hence, the RCT is least generalizable to the practice setting. Yet, the RCT is the only type of study that is being considered in high profile reviews that make a difference in the healthcare industry. For more on this topic, read Anthony Rosner’s article "Fables or foibles: inherent problems with RCTs. JMPT 2003; 26(7):460-7.

Hopefully your careful consideration of the material above will help shed some light on the content of the literature reviews you perform and the articles you read. Unfortunately, scientific quality is often achieved at the expense of our ability to use the information to make decisions in practice. The evidence base for chiropractic may be considered weak in terms of scientific quality, but you should be able to find a rich source of case studies and practice-based research that is useful. As you will see, the importance of any individual study might be small, but with several studies taken together, progress is made over time.