Poking Holes in the Evidence for Acupuncture

ball3

Throughout this entire discourse on acupuncture and needling for painful problems, Dunning et al have shown an inability or unwillingness to address central points in our argument against the effectiveness of acupuncture and needling. In addition to arriving at vastly different conclusions regarding the current trial data, Dunning and colleagues have routinely shifted the focus of the debate and presented arguments that do not relate directly to the primary issue of acupuncture’s benefits. Further, Dunning et al have consistently demonstrated a fundamental misunderstanding regarding the philosophy and implementation of evidence based practice. Their present reply is no different as they commit several critical errors in their conceptualization of evidence based practice.

For those who have missed out on earlier parts of this discussion, you can catch up below:

The Evidence for Acupuncture by James Dunning, Raymond Butts, Thomas Perreault and Firas Mourad

Acupuncture Effect Not Clinically Meaningful by Kenneth Venere and Kyle Ridgeway

Effectiveness of Acupuncture for Pain and Knee Osteoarthritis: the Evidence in Full by Dunning et al

Needle in the Hay – A Reply to Dunning et al by Venere & Ridgeway

Evidence-based practice: external clinical evidence can inform, but can never replace, individual clinical expertise by Dunning et al

What are the Issues with Therapeutic or Trigger Point Dry Needling? 9 Considerations to Ponder by Kyle Ridgeway

Debate and argument are essential to the development of strong critical thinking and provides a catalyst for the growth and evolution of science. We hope that the readership finds the recent exchanges on acupuncture and dry needling as useful as we have.


Evidence Based Practice Is Not A Math Equation

There is perhaps no greater example of Dunning et al’s erroneous interpretation of evidence based practice than when they choose to refer to evidence based practice in terms of three equal “pillars” or as a “a three legged stool”. Dunning et al cement their misunderstanding when they misinterpret quotes from David Sackett on the roles of clinical expertise, evidence and patient values. The idea that evidence based practice consists of “pillars” that are similar, if not equal in their purpose and can be cherry picked at will is grossly incorrect. Evidence, clinical experience, and patient values all have separate, although at times overlapping, roles in the philosophy of evidence based practice. By quoting Sackett in the fashion they do, Dunning et al seek to somehow imply that evidence discounting the existence of a clinically meaningful effect can somehow be superseded by clinical experience of the contrary. If that is the case, what is the role of evidence? Such an assertion implies there is actually no need at all for research, especially if it contradicts previous held beliefs or clinical conclusions.

Specifically, clinical experience, should have no significant evidential role in determining efficacy or effectiveness with the exception of being a hypothesis generator in the presence of observational studies and randomized trials. This is because, after all, “empirical science is our most systematic and reflective effort at learning from experience” (Broadbent 2013). Dunning and colleagues have created a false definition of evidence based practice whereby clinical experience and published research evidence serve similar purposes and are somehow competing for supremacy in demonstrating whether or not a treatment works. This is absolutely not in line with the philosophy of evidence based practice.

Individual clinical experience is rightfully placed at the bottom of the evidence hierarchy due to the incredible amount of confounding variables and bias present when a clinician observes a treatment as being effective. Clinical experience’s value does not come from its evidential role, but from the development of expertise in interpreting research, integrating evidence with unique patient circumstances and other things. The role of clinical experience and expertise is better conceptualized as per Jeremy Howick: “Evidence Based Medicine requires clinical expertise for producing and interpreting evidence, performing clinical skills, and integrating the best research evidence with patient values and circumstances.” To be clear, we are absolutely not claiming that clinical expertise is useless, but to use it in an evidential role is squandering its value.

In his editorial “Medical reversal: What are you doing wrong for your patient today?”, Daniel Fatovich details the phenomena of medical reversal, where newer evidence of better quality refutes current clinical practices and how many treatments that have appeared to be effective later to turn out to be anything but. The issue of ineffective interventions pervading clinical practice is well storied in medicine and physical therapy. Treatments become entrenched in clinical practice due to observed outcomes and passionate advocates of the treatment prior to being properly vetted by randomized controlled trials. Fatovich argues that many issues surrounding medical reversal “reflect the prevailing paradigm of our implicit belief in expertise, but experts are often wrong(Fatovich 2013).  After underwhelming or ineffective results are shown in comparative trials, the ineffective treatment often persists in clinical practice due to passionate experts (often with significant vested interests) touting the benefit of a particular intervention. Of note, David Sackett later called for the compulsory retirement of experts and wrote “there are still far more experts around than is healthy for the advancement of science(Sackett 2000)

The idea that one needs to practice in a particular area, produce relevant randomized controlled trials, or receive formal training in a particular intervention to be able to effectively and accurately interpret the available body of evidence is not only absurd, but insulting. While we make no claim to have formal training in acupuncture or to have used it clinically, we do not understand how this contributes meaningfully to a discussion regarding the evidence for acupuncture. The point is an unnecessary ad hominem, an unwarranted appeal to authority, and an attempt to argue from a stance of perceived authority.

This type of tactic fosters a culture where only a select few are “allowed” to discuss the current state of the literature. This stymies scientific progress by creating a false dichotomy whereby only certain clinical authorities are qualified to comment on or critique research.  Dunning et al’s stance is unacceptable and unlike the approach of any scientifically grounded discipline where there are experts, but no authorities. For example, we have no present training in homeopathy, have not conducted any trials studying homeopathy for pain, but we are fairly confident that it is not effective and nothing more than the elaborate prescription of water. To dismiss this critique due to our lack of experience with homeopathy would be absurd. Further, it must be asked: are physical therapists then not qualified to critique the research surrounding the general medical management of musculoskeletal conditions? The overuse of imaging? Health services and public policy? Spinal surgery? Reiki? Craniosacral therapy? The list goes on.

We are profoundly disappointed by such a tactic and view it as a poor attempt at counter argument. Scientific literacy and critical appraisal of literature are not unique to any one area of practice, but in fact a universal commonality of a medical profession. This argument serves as another example of Dunning et al attempting to shift the goal posts away from their continued misinterpretation and misrepresentation of the totality of evidence regarding acupuncture’s benefits. We refuse to tolerate the implications that our practice settings and lack of formal training in a specific intervention limits our ability to interpret or discuss the research on a particular intervention.


Acupuncture Is Theatrical Placebo

In regards to Dunning et al’s stance that research supports acupuncture as an effective treatment for pain, we again strongly disagree. Our conclusion is that the clinical trial evidence overwhelming illustrates that acupuncture is not an efficacious nor effective treatment for painful problems. This conclusion is congruent with the highest quality evidence available, which is detailed below and has been written on extensively in our previous replies. We do agree with Dunning et al in that the acupuncture literature, for various reasons, is applicable to the general discussion of dry needling. As we have stated previously, there is much to be learned from the thousands of trials on acupuncture. The idea that acupuncture is effective for the treatment of pain, however, is not one of them.

Disappointingly, Dunning et al have continued to ignore our primary arguments against the data they misrepresent in their rebuttals. In lieu of addressing the specifics, Dunning et al have made broad, irrepresentative statements and attempted to overwhelm with citation volume. We presented detailed descriptions and analysis of the trial designs, limitations, and data of many of their citations. A significant amount of the literature cited by Dunning et al (and others we reference) unambiguously contradict such claims as “a large number of studies have demonstrated the effectiveness of acupuncture for reducing pain and disability”.

We see nowhere in their present reply where Dunning et al address the fact that they cited Vas (2012), Khosrawi (2012), Kumnerdee (2010), Yang (2009), Sim (2011), Melchart (1999), Fernández-Carnero (2010), Gonzalez-Perez (2012), Green (2008), Tough (2009), Irnich (2001),  Irnich (2002), Cherkin (2009), Macpherson (2004), Cotchett (2010), and Itoh (2004), as strong evidence for the support of acupuncture, which as we have shown previously, are not at all indicative of acupuncture being useful in treating pain. This can not be seen as anything but a misinterpretation and subsequent misrepresentation of the available evidence.

Given the volume of acupuncture literature available, the totality of evidence must be taken into account. Thus, we see little reason to discuss individual trials beyond how they demonstrate the problems of interpretive bias or weak trial design prone to finding falsely positive results. Dunning et al again fail to acknowledge the specific data in the Vickers (2012), Manheimer (2010), and Corbett (2013) systematic reviews and instead choose to focus on broad quotes that superficially support their position. We must re-iterate the fact that Vickers (2012) found the difference between sham acupuncture and true acupuncture to be a mere 5 points on a 100 point scale, which is a clinically meaningless difference that can likely be attributed to statistical noise. The Corbett (2013) network meta-analysis, in high quality trials, demonstrated that real acupuncture elicited a 5 point greater decrease in pain on a 100 point scale as compared to sham acupuncture, echoing the underwhelming results of Vickers (2012). Finally, Manheimer (2010) found that “Sham-controlled trials show statistically significant benefits; however, these benefits are small, do not meet our pre-defined thresholds for clinical relevance, and are probably due at least partially to placebo effects from incomplete blinding.” Further subgroup analyses revealed a 14.5 point improvement over wait list controls which falls short of a previously established minimally clinically important improvement of 19.9 by Tubuch (2005). Comparing nearly any intervention to a wait list control group is likely to result in some improvements in self-reported symptoms, even still, acupuncture failed to reach clinical significance.

The three additional systematic reviews cited by Dunning et al in their present reply (Cao (2012), Ezzo (2001) and White (2007)) reviewed a limited number of sham controlled acupuncture trials, all of which were later included in the more current Vickers (2012), Manheimer (2010) and Corbett (2013) reviews as well as the Derry (2006), Madsen (2009) and Ernst (2010) reviews which Dunning et al have again failed to address. As we have shown several times, the actual data in these reviews shows that when acupuncture is compared to an adequate sham the results are unequivocal in that there is no meaningful difference and acupuncture often fails to reach a minimally clinically important improvement when compared to wait-list controls (as seen in the Corbett (2013) review). Madsen (2009) also found that the type of sham acupuncture, whether shallow or non-penetrating, has no meaningful effect on the efficacy of acupuncture for pain suggesting no specific effect to needling. Kaptchuk (2006) has stated ‘‘acupuncture should not hide behind the excuse that anything you do with a needle, including waving it, is a form of acupuncture.” As O’Connell (2009) writes, “it does not matter where the needles are placed, how deep they are inserted, or whether they are inserted at all.” Therefore, we see the argument that comparing verum acupuncture to sham acupuncture as being polluted due to sham acupuncture potentially being “active” as a moot point. It is also quite the logical leap to justify practice on. If the sham acupuncture was in fact “active” then what is the specific application and mechanistic difference between the interventions? If both interventions were actually active and effective, results from both groups would be larger and more compelling.

A further example of Dunning et al citing literature as “support” for acupuncture without any meaningful appraisal of the actual research includes their ringing endorsement of the GERAC trial for patients with low back pain by Haake et al. The grossly inappropriate conclusions of this trial have been thoroughly derided by several diffierent authors (Goldacre 2007, Ernst (2008), Wand (2008) and (O’Connell 2009)) and we wish to direct readers to these well written pieces.

To quote Wand (2008):

The authors chose to compare the effectiveness of acupuncture with conventional medical care in a sample with long-standing low back pain. It is highly likely that conventional therapy had already failed in most of this population, introducing the potential for a considerable nocebo effect in those assigned to this group. In addition, acupuncture was a novel intervention for participants and is known to be associated with particularly strong non- specific effects, a situation likely to lead to a strong placebo response in those receiving acupuncture. To mitigate these threats to the study’s internal validity, the authors sensibly chose to include a sham acupuncture group. The results clearly show no difference between true and sham acupuncture, and the reasonable interpretation of this result is that acupuncture is not effective beyond placebo. Instead, the authors state that this is “unequivocal” evidence that acupuncture is superior to conventional care and claim that the results suggest a new and as yet unspecified mechanism of action of acupuncture. While most readers would not begrudge the authors a speculative sentence or two on this issue, to make this the major thrust of the discussion is both misleading and unscientific.

We must re-iterate that we are not interested in discussing surrogate outcome measures such as fMRI changes, concentration of certain chemicals, and other physiologic measures in the absence of meaningful results in actual clinical endpoints. Such measures, while interesting, do not provide robust support for the use of an intervention that’s shown no meaningful clinical benefit. Given that needling is a neurologic input in a treatment environment it is expected to find some changes in physiologic measures, neurologic activity, and brain activity, but this is not a primary support that the intervention is actually having meaningful effect. Nearly any input will result in some measured physiologic change from the baseline state, even if you are a dead salmon.

Further, there is no point in presently discussing how acupuncture research alone compares to studies of other interventions such as core stabilization, NSAIDs and deep neck flexor training. While effect sizes are highly important in clinical research, it is in an error to directly compare effect sizes between trials of different interventions using different measurements as a means of directly ascertaining differences in magnitude of effects between those interventions. Comparisons of effect sizes between studies is much more complicated, and is dependent on intervention, outcomes measured, and how effect size was caclulated. We are sure that there are issues to be discussed surrounding the research and implementation of other interventions. But, the current discussion is focused on the specific efficacy and effectiveness of acupuncture in treating pain and our replies are solely focused this issue. By introducing these comparisons, Dunning et al again shift the focus from the primary question “is acupuncture or needling effective for the treatment of pain?” to other topics that cloud the actual debate, and do not support their conclusion. Even if acupuncture had a larger magnitude of effect than other interventions, that does support acupuncture as having any sort of efficacy.  


In Closing

As it stands, evidence from high quality trials (subsequently systematically reviewed and analyzed) unambiguously provides the most reliable information on a treatment’s efficacy or effectiveness. This is not up for debate and can not possibly be superseded by clinical expertise’s evidential role (Howick 2011). Now, this is by no means a lambasting of clinical experience and expertise as useless to clinical practice. Quite the contrary. But, Dunning et al have shown numerous times that they are unable, or unwilling, to accurately interpret acupuncture literature. Richard Feynman describes science as a culture of doubt and the review data above gives more than enough reason to doubt the usefulness of acupuncture across many conditions. To attempt to rationalize the negative data and the continued practice of needling on the lofty assumption that sham acupuncture is often “active” is borderline absurd and not how a doctoring profession should justify its clinical practice.

Hopefully, the readership gleans some benefit from this discussion while recognizing many of the issues in the arguments of Dunning et al. Given Dunning et al’s continuing education focus we do not expect them to change their view of needling for painful problems. Those less familiar with needling, or on the fence, should now have a better understanding of the totality of the available evidence. Finally, we commend Dunning et al for directly stating they teach needling courses (although it likely did not bear repeating), but will continue to highlight how such a situation can lead to improper interpretation of the research evidence albeit not necessarily purposefully. Dunning et al’s attempts at argument illustrate such a fact given the inaccuracy of their statements compared to their citations. This discourse serves to highlight the inherent challenges present when individuals who believe a priori an intervention is effective, and also financially benefit from teaching said intervention, are confronted with refutational data and forced to answer pointed critique. There are perils associated with a lack of personal equipoise and we feel this discourse has illustrated them clearly.

In conclusion, clinical experience alone does not prove effectiveness and subsequently clinical experience can not override well controlled studies on efficacy and effectiveness. Evidence based practice is multifaceted and complex requiring clinical expertise in both clinical and cognitive skills to apply the whole of scientific knowledge to the treatment of an individual. Acupuncture research illustrates that it is not specifically efficacious or effective in the treatment of painful problems, including knee osteoarthritis.

Kenneth Venere PT, DPT

Kyle Ridgeway PT, DPT

References

Broadbent A. Philosophy of Epidemiology. Palgrave Macmillan; 2013.

Cao L, Zhang XL, Gao YS, Jiang Y. Needle acupuncture for osteoarthritis of the knee. A systematic review and updated meta-analysis. Saudi Med J. May 2012;33(5):526-532.

Cherkin DC, Sherman KJ, Avins AL, Erro JH, Ichikawa L, Barlow WE, et al. A randomized trial comparing acupuncture, simulated acupuncture, and usual care for chronic low back pain. Arch Intern Med. 2009;169(9):858-66.

Corbett MS, Rice SJ, Madurasinghe V, Slack R, Fayter DA, Harden M, et al. Acupuncture and other physical treatments for the relief of pain due to osteoarthritis of the knee: network meta-analysis. Osteoarthritis Cartilage. 2013;21(9):1290-8.

Cotchett MP, Landorf KB, Munteanu SE. Effectiveness of dry needling and injections of myofascial trigger points associated with plantar heel pain: a systematic review. J Foot Ankle Res. 2010;3:18.

Derry CJ, Derry S, McQuay HJ, Moore RA. Systematic review of systematic reviews of acupuncture published 1996-2005. Clin Med. 2006 Jul-Aug;6(4):381-6. PubMed PMID: 16956145.

Ernst E, Lee MS, Choi TY. Acupuncture: does it alleviate pain and are there serious risks? A review of reviews. Pain 2011;152:755–64

Ernst E. Acupuncture Ineffective, Attention Effective?. Arch Intern Med. 2008;168(5):551. doi:10.1001/archinte.168.5.551.

Ezzo J, Hadhazy V, Birch S, et al. Acupuncture for osteoarthritis of the knee: a systematic review. Arthritis Rheum. Apr 2001;44(4):819-825.

Fatovich DM. Medical reversal: What are you doing wrong for your patient today?. Emerg Med Australas. 2013 Feb;25(1):1-3. PubMed PMID: 23379445.

Fernandez-Carnero J, La Touche R, Ortega-Santiago R, Galan-del-Rio F, Pesquera J, Ge HY, et al. Short-term effects of dry needling of active myofascial trigger points in the masseter muscle in patients with temporomandibular disorders. J Orofac Pain. 2010;24(1):106-12.

Goldacre B. Medicalisation – don’t take it lying down. 2007; Available from: http://www.badscience.net/2007/09/542/

Gonzalez-Perez LM, Infante-Cossio P, Granados-Nunez M, Urresti-Lopez FJ. Treatment of temporomandibular myofascial pain with deep dry needling. Med Oral Patol Oral Cir Bucal. 2012;17(5):e781-5.

Green S, Buchbinder R, Hetrick S. Acupuncture for shoulder pain. Cochrane Database Syst Rev. 2008

Haake M, Muller HH, Schade-Brittinger C, et al. German Acupuncture Trials (GERAC) for chronic low back pain: randomized, multicenter, blinded, parallel-group trial with 3 groups. Arch Intern Med. Sep 24 2007;167(17):1892-1898.

Howick J. The Philosophy of Evidence-based Medicine. BMJ Books; 2011.

Irnich D, Behrens N, Gleditsch JM, Stor W, Schreiber MA, Schops P, et al. Immediate effects of dry needling and acupuncture at distant points in chronic neck pain: results of a randomized, double-blind, sham-controlled crossover trial. Pain. 2002;99(1-2):83-9.

Irnich D, Behrens N, Molzen H, Konig A, Gleditsch J, Krauss M, et al. Randomised trial of acupuncture compared with conventional massage and “sham” laser acupuncture for treatment of chronic neck pain. BMJ. 2001;322(7302):1574-8.

Itoh K, Katsumi Y, Kitakoji H. Trigger point acupuncture treatment of chronic low back pain in elderly patients–a blinded RCT. Acupunct Med. 2004;22(4):170-7.

Kaptchuk TJ. Effect of interpretive bias on research evidence. BMJ  Br Med J. 2003;326(7404):1453-1455. doi:10.1136/bmj.326.7404.1453.

Kaptchuk, T. J. (2006). A plastic eagle feather is a placebo. Rapid response to, Kaptchuk, T. J., Stason, W. B., Davis, R. B., Legedza, A. T. R., Schnyer, R. N., Kerr, C. E., et al. (2006). Sham device versus inert pill: Randomised controlled trial of two placebo treatments. British Medical Journal, 332, 7538,391-397.

Khosrawi S, Moghtaderi A, Haghighat S. Acupuncture in treatment of carpal tunnel syndrome: A randomized controlled trial study. J Res Med Sci. 2012;17(1):1-7.

Kumnerddee W, Kaewtong A. Efficacy of acupuncture versus night splinting for carpal tunnel syndrome: a randomized clinical trial. J Med Assoc Thai. 2010;93(12):1463-9.

MacPherson H, Thorpe L, Thomas K, Campbell M. Acupuncture for low back pain: traditional diagnosis and treatment of 148 patients in a clinical trial. Complement Ther Med. 2004;12(1):38-44.

Madsen MV, Gøtzsche PC, Hróbjartsson A. Acupuncture treatment for pain: systematic review of randomised clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups. BMJ. 2009;338(January 2008):a3115. doi:10.1136/bmj.a3115.

Manheimer E, Cheng K, Linde K, Lao L, Yoo J, Wieland S, et al. Acupuncture for peripheral joint osteoarthritis. Cochrane Database Syst Rev. 2010(1):CD001977

Manheimer E, Linde K, Lao L, Bouter LM, Berman BM. Meta-analysis: acupuncture for osteoarthritis of the knee. Ann Intern Med. 2007;146(12):868-77.

Melchart D, Linde K, Fischer P, White A, Allais G, Vickers A, et al. Acupuncture for recurrent headaches: a systematic review of randomized controlled trials. Cephalalgia. 1999;19(9):779-86; discussion 65.

O’Connell NE, Wand BM, Goldacre B. Interpretive bias in acupuncture research?: A case study. Eval Health Prof. 2009;32(4):393-409. doi:10.1177/0163278709353394.

Sackett DL. The sins of expertness and a proposal for redemption. BMJ : British Medical Journal. 2000;320(7244):1283.

Sim H, Shin BC, Lee MS, Jung A, Lee H, Ernst E. Acupuncture for carpal tunnel syndrome: a systematic review of randomized controlled trials. J Pain. 2011;12(3):307-14.

Tough EA, White AR, Cummings TM, Richards SH, Campbell JL. Acupuncture and dry needling in the management of myofascial trigger point pain: a systematic review and meta-analysis of randomised controlled trials. Eur J Pain. 2009;13(1):3-10.

Tubach F, Ravaud P, Baron G, et al. Evaluation of clinically relevant changes in patient reported outcomes in knee and hip osteoarthritis: the minimal clinically important improvement. Ann Rheum Dis. 2005;64(1):29-33. doi:10.1136/ard.2004.022905.

Vas J, Aranda JM, Modesto M, Benitez-Parejo N, Herrera A, Martinez-Barquin DM, et al. Acupuncture in patients with acute low back pain: a multicentre randomised controlled clinical trial. Pain. 2012;153(9):1883-9.

Vickers AJ, Cronin AM, Maschino AC, Lewith G, MacPherson H, Foster NE, et al. Acupuncture for chronic pain: individual patient data meta-analysis. Arch Intern Med. 2012;172(19):1444-53.

Wand BM, O’Connell N. German Acupuncture Trials for Chronic Low Back Pain. Arch Intern Med. 2008;168(9):1011. doi:10.1001/archinte.168.9.1011-a.

White A, Foster NE, Cummings M, Barlas P. Acupuncture treatment for chronic knee pain: a systematic review. Rheumatology (Oxford). Mar 2007;46(3):384-390.

Yang CP, Hsieh CL, Wang NH, Li TC, Hwang KL, Yu SC, et al. Acupuncture in patients with carpal tunnel syndrome: A randomized controlled trial. Clin J Pain. 2009;25(4):327-33.

Photo courtesy of flickr user Brent Schneeman

Advertisements

2 thoughts on “Poking Holes in the Evidence for Acupuncture

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s