Dunning et al write that the “the purpose of this article is to directly respond to these claims in a public forum”, however after reading their reply it would seem their idea of directly responding to the claims in our article are to craft an argument from authority, attempt to demonstrate proof by verbosity, mischaracterize the points we made and to shift the goal posts away from the actual acupuncture data towards ad hominem attacks on our personal and professional credibility. We see no part of Dunning et al’s reply that adequately addresses the crux of our PT in Motion article -- The fact that Dunning et al misrepresented the findings of the Manheimer (2010) and Vickers (2012) systematic reviews, which include many of the trials Dunning et al cite in their original article and their recent blog post.
“Venere and Ridgeway claim acupuncture is not an effective treatment strategy for pain; however, as already detailed and well referenced in the 2014 literature review by Dunning et al, a large number of studies have demonstrated the effectiveness of acupuncture for reducing pain and disability … Venere and Ridgeway have provided nothing more than their own personal opinions on the subject of acupuncture induced hypoalgesia”
We find it interesting that Dunning lambasted our piece as “opinion”, but Dunning et al’s original PT in Motion piece, their present reply, and their narrative review would also qualify as a collective opinion on acupuncture literature. We are not sure why Dunning et al attempts to characterize an informed opinion of the acupuncture literature as a negative or that it somehow takes away from the actual data presented when they are also simply producing opinions of their own. It is of note that we have no “skin in the game” of dry needling and acupuncture, other than being passionate advocates for science and evidence based practice in physical therapy. Dunning et al however, have significant vested interests in the success of needling and other complementary and alternative medicine practices such as cupping. Whether implicit or explicit, there is great potential for interpretive bias when significant conflicts of interest are present (Cook 2011, Kaptchuk 2003 and O’Connell 2009). This is evident in their selective appraisal of the acupuncture literature.
Dunning et al criticize our reply for “narrowly focused their claims on the partial findings of a single publication and supported their position with just 5 unique references.” which not only mischaracterizes our article, but attempts to shift the goal posts away from directly addressing our claims and instead argues from an untenable position of proof by verbosity. Dunning et al have grandstanded their 104 citations on twitter as some sort of indicator that their position is more sound. This attempt at overwhelming by citation often coincides with a common tactic in the acupuncture and other complementary and alternative medicine circles to cherry pick individual “positive” (we use this term loosely) trials while ignoring the quality of the individual trials themselves and the totality of the evidence for a particular intervention. Our article concisely focused on the misleading claims that Dunning et al made in their PT in Motion article and presented data from the articles they cited. To assume that the number of references is somehow an indicator of the quality of the content is misguided and absurd. Quality is more important than quantity. It would be redundant to cite the many individual trials that were already included in the cited systematic reviews and meta analyses.
The unambiguous results of the following systematic reviews (and systematic reviews of systematic reviews) that analysed thousands of subjects in acupuncture trials do not qualify as opinion: Derry (2006), Ernst (2010), Madsen (2009) all found acupuncture to be nothing more than a theatrical placebo. The individual trials Dunning et al cite are worth a closer look, as it would appear they have made significant errors in their interpretation. Dunning et al appear to have a history of misrepresenting or misinterpreting the evidence for needling, as their original narrative review was temporarily removed from publication after concerns were brought up to the editor of the associated journal regarding their claims made regarding knee osteoarthritis and acupuncture. It is also important to consider the problems with many of the individual trials conducted on acupuncture (and later included in systematic reviews) such as bias (Vickers 1998 and O’Connell 2009), and poor methodological quality (Derry 2006) which makes them extremely prone to false positives (Ionaddis 2013).
Dunning et al try to justify the results of Manheimer (2010) by hanging their hats on a secondary analysis that we again feel Dunning et al have misrepresented. Dunning et al claim “a subgroup analysis found the effects of verum acupuncture were clinically relevant when compared to several active treatments and wait-list controls, and the authors suggest that patients with osteoarthritis will find meaningful benefits from acupuncture.”
Let us take a look the actual data. Manheimer (2010) state:
a secondary analysis versus a waiting list control, acupuncture was associated with statistically significant, clinically relevant short-term improvements in osteoarthritis pain (-0.96, -1.19 to -0.72; 14.5 point greater improvement than sham on 100 point scale; absolute percent change 14.5%; relative percent change 29.14%; 4 trials; 884 participants) and function (-0.89, -1.18 to -0.60; 13.0 point greater improvement than sham on 100 point scale; absolute percent change 13.0%; relative percent change 25.21%). In the head-on comparisons of acupuncture with the 'supervised osteoarthritis education' and the 'physician consultation' control groups, acupuncture was associated with clinically relevant short- and long-term improvements in pain and function. In the head on comparisons of acupuncture with 'home exercises/advice leaflet' and 'supervised exercise', acupuncture was associated with similar treatment effects as the controls. Acupuncture as an adjuvant to an exercise based physiotherapy program did not result in any greater improvements than the exercise program alone. Information on safety was reported in only 8 trials and even in these trials there was limited reporting and heterogeneous methods.
When appraising the data, we see a 14.5 point improvement over wait list controls. One would expect that nearly any treatment would improve the subjective report of pain in such a limited fashion when compared to a wait-list control. In our PT in Motion article, we cited Tubuch 2005 as having found the minimal clinically important improvement for a 0-100 scale to be 19.9 in patients with knee osteoarthritis, thus calling into question the clinical relevance of these secondary analysis results. When compared to more active controls such as home exercise, acupuncture showed similar treatment effects as the controls and was not responsible for a greater change in outcome when compared to traditional exercise based physiotherapy programs.
We must also bring attention to the significant concerns over subgroup and secondary analyses when the actual pooled effect is null. When the pooled effect size of a treatment is small, as Manheimer (2010)’s analysis shows, this is when secondary analysis and subgroups are the least plausible. This is because, as Hancock et al state, “the only way that a proportion of patients can receive a large effect is if the treatment is actually harmful (compared with the control condition) for other patients.” (Hancock 2009). With the results of the primary analysis and the above qualifications in mind, we must ask, does a secondary analysis showing a 14.5 point improvement in pain over waitlist controls qualify as strong support for the use of acupuncture? We think not.
Dunning et al also neglect to directly address the results of the Vickers (2012) trial we highlighted in our original article which showed that the difference between sham acupuncture and true acupuncture is a mere 5 points on a 100 point scale, undoubtedly a meaningless clinical effect. Instead of acknowledging this, Dunning et al vaguely claim that several of the included trials may have had a sham intervention that was likely active, while ignoring the fact that it was not a requirement that all included trials contain a sham/placebo group (which would serve to inflate the effect size of acupuncture) and the author’s stating that there is considerable heterogeneity in the included trials. Both of which are bigger threats to the validity of the systematic review and it’s ability to show acupuncture as an effective treatment.
In their reply, Dunning et al have seemingly cited articles without vetting them for quality and without knowledge of the actual results because many of their citations directly contradict their central point that acupuncture is effective for those in pain. In addition to the misinterpretations of the evidence above, Dunning et al cite Vas (2012) as evidence acupuncture is effective for back pain, but the authors state “there was no difference among the 3 acupuncture modalities, which implies that true acupuncture is not better than sham or placebo acupuncture.” Another example is the meta analysis for acupuncture in Knee OA conducted by Manheimer (2007) that concludes “Sham-controlled trials show clinically irrelevant short-term benefits of acupuncture for treating knee osteoarthritis.” In the Corbett (2013) network meta-analysis, in trials of high quality, real acupuncture elicited a 5 point greater decrease in pain on a 100 point scale as compared to sham acupuncture, echoing the clinically meaningless results of Vickers (2012).
Further examples include Khosrawi (2012) showing a 16% difference between acupuncture and sham after four weeks in GSS score, Kumnerdee (2010) claiming it is an efficacy trial despite having very poor internal validity (with no relevant control group) and no relevant effects on functional status scale and symptom severity scale results, Yang (2009) did not have a sham acupuncture group so the claims on the true effect of acupuncture can not be ascertained, Sim (2011) in their systematic review conclude "The existing evidence is not convincing enough to suggest that acupuncture is an effective therapy for carpal tunnel syndrome.”, Melchart (1999) state in their systematic review “The eight trials comparing acupuncture and other treatment forms had contradictory results [and] the quality and amount of evidence is not fully convincing”, Fernández-Carnero (2010) looked at the effects of needling only 5 minutes after treatment which is a study design of limited clinical utility (Cook 2011), Gonzalez-Perez (2012) had no comparison group which again shows the poor quality literature Dunning et al cite in support of acupuncture, Green (2008) in their systematic review write “The improvements with acupuncture for pain and function were about the same as the effects of receiving a fake therapy for 2 to 4 weeks.” Tough (2009) write “Whilst the result of the meta-analysis of needling compared with placebo controls does not attain statistically significant, the overall direction could be compatible with a treatment effect of dry needling on myofascial trigger point pain.” which is hardly a ringing endorsement. Irnich (2002)’s results show a 1.0mm (of 100mm) reduction in pain which is clinically meaningless, Irnich (2001) showed acupuncture was no better than sham laser for reducing pain, Cherkin (2009) write “It remains unclear whether acupuncture or our simulated method of acupuncture provide physiologically important stimulation or represent placebo or nonspecific effects.”, Macpherson (2004) diagnosed patients with low back pain as caused by Qi and Blood stagnation which is a pseudoscientfic fantasy, Cotchett (2010) conclude in their systematic review that “There is limited evidence for the effectiveness of dry needling and/or injections of MTrPs associated with plantar heel pain. However, the poor quality and heterogeneous nature of the included studies precludes definitive conclusions being made.” and of note, Cotchett (2013) later produced a randomized controlled trial published in the PTJ, that Dunning et al neglect to cite, that showed a number needed to harm of 3 and number needed to treat of 4, that is for every 3 patients treated with dry needling, one will experience an adverse event, and for every four treated, one will experience a positive outcome. Unfortunately, that positive outcome is limited to clinically irrelevant effects when receiving dry needling for heel pain. Itoh (2004) also had a small number of subjects with no sham group, as such no firm conclusions can be drawn regarding the benefits of acupuncture separate from placebo due to the study’s poor quality. One does not need formal training in acupuncture to see that the studies cited are of poor quality and often directly contradict the central point of Dunning et al.
Another common way to attempt justify the use of ineffective treatments is to cite numerous surrogate outcome and basic science trials. While these studies cited by Dunning et al are intriguing, in the context of the overwhelming about of clinical trial data that suggests acupuncture has no meaningful effect on actual clinical end points such as pain and disability as noted above, they ultimately do not serve as a justification for implementing needling into clinical practice. To quote Colquhoun and Novella (2013)“We see no point in discussing surrogate outcomes, such as functional magnetic resonance imaging studies or endorphine release studies, until such time as it has been shown that patients get a useful degree of relief. It is now clear that they do not.”
“To our knowledge, Venere and Ridgeway have no formal training in acupuncture, would not be considered as academic or clinical experts in the use of acupuncture for the treatment of pain and disability in musculoskeletal conditions—i.e. they work in home health and acute care settings, respectively—and have yet to publish a single article in a peer-reviewed journal on the topic. Therefore, considering the 3 pillars of evidence-based practice as originally put forward by Sackett, Venere and Ridgeway have provided nothing more than their own personal opinions on the subject of acupuncture induced hypoalgesia.”
We take issue with the idea that one needs formal training or to have published research within the field of acupuncture to critically appraise and make informed decisions from the available literature. This is an ad hominem attack aimed at denigrating our personal and professional credibility, as opposed to directly addressing the claims and data we presented. We would hope that Dunning et al, who have a prominent voice in the profession as educators, would be of higher character than to resort to such tactics. Of note, Dunning is an outspoken critic of the chiropractic subluxation model on twitter, yet as far as we can tell he has never received formal training in chiropractic or the subluxation model. However, we would never use this fact to question his ability to appraise and interpret the biological plausibility, efficacy or effectiveness of such a treatment approach. Nor are physical therapists criticized for making claims about the benefits of surgery or opioids for chronic pain, despite physical therapists lacking any sort of formal training in surgery or the prescription of medications. This statement on our areas of practice and absence of formal training in acupuncture are irrelevant and offensive.
We are also not sure the relevance of the reference to evidence based practice or the idea that we have provided nothing more than our personal opinions on the subject. After all, Dunning’s original PT in Motion article and this subsequent blog post are nothing more than opinion. This is not an issue of formal training in acupuncture or a particular area of practice, but the ability to accurately interpret the best available evidence and integrate it into clinical practice -- which is the central philosophy of Sackett’s evidence based practice.
In closing, we would again like to quote Colquhoun and Novella (2013):
“The best controlled studies show a clear pattern, with acupuncture the outcome does not depend on needle location or even needle insertion. Since these variables are those that define acupuncture, the only sensible conclusion is that acupuncture does not work. Everything else is the expected noise of clinical trials, and this noise seems particularly high with acupuncture research. The most parsimonious conclusion is that with acupuncture there is no signal, only noise.”
It’s time to move on.
Kenneth Venere PT, DPT
Kyle Ridgeway PT, DPT
References
Cherkin DC, Sherman KJ, Avins AL, Erro JH, Ichikawa L, Barlow WE, et al. A randomized trial comparing acupuncture, simulated acupuncture, and usual care for chronic low back pain. Arch Intern Med. 2009;169(9):858-66.
Colquhoun D, Novella SP. Acupuncture is theatrical placebo. Anesth Analg. 2013;116:1360-1363.
Cook C, Sheets C. Clinical equipoise and personal equipoise: two necessary ingredients for reducing bias in manual therapy trials. J Man Manip Ther. 2011;19(1):55-57. doi:10.1179/106698111X12899036752014.
Cook C. Immediate effects from manual therapy: much ado about nothing? J Man Manip Ther. 2011;19(1):3–4.
Corbett MS, Rice SJ, Madurasinghe V, Slack R, Fayter DA, Harden M, et al. Acupuncture and other physical treatments for the relief of pain due to osteoarthritis of the knee: network meta-analysis. Osteoarthritis Cartilage. 2013;21(9):1290-8.
Cotchett MP, Landorf KB, Munteanu SE. Effectiveness of dry needling and injections of myofascial trigger points associated with plantar heel pain: a systematic review. J Foot Ankle Res. 2010;3:18.
Cotchett MP, Munteanu SE, Landorf KB. Effectiveness of Trigger Point Dry Needling for Plantar Heel Pain: A Randomized Controlled Trial. Phys Ther [Internet]. 2014; Available from: http://www.ncbi.nlm.nih.gov/pubmed/24700136
Derry CJ, Derry S, McQuay HJ, Moore RA. Systematic review of systematic reviews of acupuncture published 1996-2005. Clin Med. 2006 Jul-Aug;6(4):381-6. PubMed PMID: 16956145.
Ernst E, Lee MS, Choi TY. Acupuncture: does it alleviate pain and are there serious risks? A review of reviews. Pain 2011;152:755–64
Fernandez-Carnero J, La Touche R, Ortega-Santiago R, Galan-del-Rio F, Pesquera J, Ge HY, et al. Short-term effects of dry needling of active myofascial trigger points in the masseter muscle in patients with temporomandibular disorders. J Orofac Pain. 2010;24(1):106-12.
Gonzalez-Perez LM, Infante-Cossio P, Granados-Nunez M, Urresti-Lopez FJ. Treatment of temporomandibular myofascial pain with deep dry needling. Med Oral Patol Oral Cir Bucal. 2012;17(5):e781-5.
Green S, Buchbinder R, Hetrick S. Acupuncture for shoulder pain. Cochrane Database Syst Rev. 2008
Hancock M, Herbert RD, Maher CG. A guide to interpretation of studies investigating subgroups of responders to physical therapy interventions. Phys Ther. 2009;89:698–704.
Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2:e124.
Irnich D, Behrens N, Gleditsch JM, Stor W, Schreiber MA, Schops P, et al. Immediate effects of dry needling and acupuncture at distant points in chronic neck pain: results of a randomized, double-blind, sham-controlled crossover trial. Pain. 2002;99(1-2):83-9.
Irnich D, Behrens N, Molzen H, Konig A, Gleditsch J, Krauss M, et al. Randomised trial of acupuncture compared with conventional massage and “sham” laser acupuncture for treatment of chronic neck pain. BMJ. 2001;322(7302):1574-8.
Itoh K, Katsumi Y, Kitakoji H. Trigger point acupuncture treatment of chronic low back pain in elderly patients–a blinded RCT. Acupunct Med. 2004;22(4):170-7.
Kaptchuk TJ. Effect of interpretive bias on research evidence. BMJ Br Med J. 2003;326(7404):1453-1455. doi:10.1136/bmj.326.7404.1453.
Khosrawi S, Moghtaderi A, Haghighat S. Acupuncture in treatment of carpal tunnel syndrome: A randomized controlled trial study. J Res Med Sci. 2012;17(1):1-7.
Kumnerddee W, Kaewtong A. Efficacy of acupuncture versus night splinting for carpal tunnel syndrome: a randomized clinical trial. J Med Assoc Thai. 2010;93(12):1463-9.
MacPherson H, Thorpe L, Thomas K, Campbell M. Acupuncture for low back pain: traditional diagnosis and treatment of 148 patients in a clinical trial. Complement Ther Med. 2004;12(1):38-44.
Madsen MV, Gøtzsche PC, Hróbjartsson A. Acupuncture treatment for pain: systematic review of randomised clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups. BMJ. 2009;338(January 2008):a3115. doi:10.1136/bmj.a3115.
Manheimer E, Cheng K, Linde K, Lao L, Yoo J, Wieland S, et al. Acupuncture for peripheral joint osteoarthritis. Cochrane Database Syst Rev. 2010(1):CD001977
Manheimer E, Linde K, Lao L, Bouter LM, Berman BM. Meta-analysis: acupuncture for osteoarthritis of the knee. Ann Intern Med. 2007;146(12):868-77.
Melchart D, Linde K, Fischer P, White A, Allais G, Vickers A, et al. Acupuncture for recurrent headaches: a systematic review of randomized controlled trials. Cephalalgia. 1999;19(9):779-86; discussion 65.
O’Connell NE, Wand BM, Goldacre B. Interpretive bias in acupuncture research?: A case study. Eval Health Prof. 2009;32(4):393-409. doi:10.1177/0163278709353394.
Sim H, Shin BC, Lee MS, Jung A, Lee H, Ernst E. Acupuncture for carpal tunnel syndrome: a systematic review of randomized controlled trials. J Pain. 2011;12(3):307-14.
Tough EA, White AR, Cummings TM, Richards SH, Campbell JL. Acupuncture and dry needling in the management of myofascial trigger point pain: a systematic review and meta-analysis of randomised controlled trials. Eur J Pain. 2009;13(1):3-10.
Tubach F, Ravaud P, Baron G, et al. Evaluation of clinically relevant changes in patient reported outcomes in knee and hip osteoarthritis: the minimal clinically important improvement. Ann Rheum Dis. 2005;64(1):29-33. doi:10.1136/ard.2004.022905.
Vas J, Aranda JM, Modesto M, Benitez-Parejo N, Herrera A, Martinez-Barquin DM, et al. Acupuncture in patients with acute low back pain: a multicentre randomised controlled clinical trial. Pain. 2012;153(9):1883-9.
Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of con- trolled trials. Control Clin Trials 1998;19:159–66.
Vickers AJ, Cronin AM, Maschino AC, Lewith G, MacPherson H, Foster NE, et al. Acupuncture for chronic pain: individual patient data meta-analysis. Arch Intern Med. 2012;172(19):1444-53.
Yang CP, Hsieh CL, Wang NH, Li TC, Hwang KL, Yu SC, et al. Acupuncture in patients with carpal tunnel syndrome: A randomized controlled trial. Clin J Pain. 2009;25(4):327-33.