Evidence Map of Acupuncture: Where We Stand and Where We Should Go
Article information
Abstract
Background
Evidence mapping presents the current status of evidence on a specific field. The "Evidence Map of Acupuncture" published in 2014 gave an overall picture of the evidence on acupuncture treatments for various conditions/diseases. In this study, evidence in 2024 for the effect of acupuncture was reassessed.
Methods
The systematic reviews (SRs) on acupuncture for the 43 conditions/diseases where evidence was previously unclear or potentially effective in the "Evidence Map of Acupuncture" 2014 were searched in the PubMed and the Cochrane Library, and included SRs up to February, 2024. The Grading of Recommendations, Assessment, Development, and Evaluations assessment in the included SRs was used for assessing the confidence level in the evidence of each condition.
Results
When compared with the results of the "Evidence Map of Acupuncture" 2014, the average number of randomized controlled trials included in a SR increased from 11 to 19.5 by 2024. However, the confidence level showed an overall decrease. The reasons for unclear evidence were mainly methodological limitations such as poor research design, small sample size and small number of studies, and the results of the 2024 reassessment did not show a significant difference in the reasons compared with the "Evidence Map of Acupuncture" 2014.
Discussion
To improve clinical evidence for acupuncture, simple repetition and increasing the number of new randomized controlled trials does not seem to be effective. To reduce redundancy, large scaled studies should be conducted, and a new critical appraisal tool for acupuncture is needed to avoid unfair evaluation of risk of bias in acupuncture research.
Introduction
Is acupuncture an effective intervention? In which conditions or diseases does acupuncture show efficacy or effectiveness? There is considerable clinical evidence for acupuncture treatment of various diseases and conditions but the debate continues.
The "Evidence Map of Acupuncture" is a report published in 2014 and was the first attempt to provide an objective picture of the scope and certainty of the effects of acupuncture [1]. It was created to provide a guide for the marginal evidence needed to offer acupuncture treatments to Veterans by the U.S. Department of Veterans Affairs [1]. Evidence mapping is an emerging research method for identifying current available evidence and finding gaps for future research. It can suggest a brief and overall summary of the current evidence, and offer depth and breadth of knowledge on a specific area, so it is usually accepted as a useful tool for researchers as well as health customers [2].
The U.S. Department of Veterans Affairs published an updated report in 2022 [3] which differs from the previous "Evidence Map of Acupuncture" 2014 report [1]. It classified evidence into benefit and no benefit, whereas the previous 2014 report categorized evidence into no effect, unclear effect, potentially effective, and positive effect [3]. The previous 2014 report categorized diseases/conditions into pain, wellness, and mental health, but the current 2022 updated report categorizes them differently. Additionally, plantar heel pain, nausea, restless leg syndrome, and blood pressure, which were included before, are missing, making it difficult to compare how the evidence for acupuncture has changed over the past decade. In this meta-epidemiological study, updates up to the current year of 2024 were made on systematic reviews (SRs) concerning acupuncture for conditions/diseases with unclear or potentially effective outcomes in the 2014 report [1]. This study aimed to determine whether the evidence for the effectiveness of acupuncture on these conditions/diseases has become conclusive. Additionally, it was evaluated whether factors highlighted as reasons for the uncertainty in deciding the level of evidence on the effectiveness of acupuncture in the 2014 report have improved through subsequent research. Data from this article may help to develop an agenda on how future clinical research and SRs on acupuncture should be conducted to improve clinical evidence of acupuncture.
Materials and Methods
1. Selection criteria
The methods of the study were largely adopted from those in the 2014 report [1]. However, the conditions/diseases of interest were limited to those included in the previous 2014 report where the evidence of effects were uncertain. The PICO model (patient/population, intervention, comparison and outcomes) for this study was as follows:
Population: Patients with any of the 43 conditions/diseases where evidence was previously unclear or potentially effective, including back pain, neck pain, surgery analgesia, postoperative pain, fibromyalgia, rheumatoid arthritis, shoulder pain, dysmenorrhea, osteoarthritis, ankle sprain, general pain, cancer pain, labor, prostatitis, plantar heel, temporomandibular, pregnancy pain, cancer adverse event, irritable bowel syndrome, rhinitis, premenstrual syndrome, gastro-intestinal disease, blood pressure, tinnitus, menopausal symptoms, exercise, dry eye, quality of life, xerostomia, erectile dysfunction, insomnia, obesity, smoking cessation, postoperative nausea and vomiting, restless legs, constipation, opiate addiction, chronic fatigue syndrome, drug addiction, depression, schizophrenia, anxiety, and posttraumatic stress disorder.
Intervention: Needle acupuncture including traditional acupuncture. Electroacupuncture, moxibustion stimulation, manual acupuncture, and microsystem acupuncture were all included.
Comparator: Sham acupuncture, no treatment, or other conventional treatments.
Outcome: Any types of patient outcomes.
Design: Only English language SRs. In cases where multiple SRs exist for the same condition/disease, we selected the latest SR with the largest numbers of included randomized controlled trials for each condition/disease. When some symptoms might include different conditions/diseases which cannot be identified as a single SR, an overview of SRs was selected.
2. Searching
For this meta-epidemiology study, a systematic search strategy for all the conditions/diseases was not adopted, instead, updated SRs for comparison with the 2014 report [1] were located interdependently for covering the 43 conditions/diseases set. If there was a Cochrane review available during that period, the Cochrane review was selected; if not, the most recently published SR was selected. The search was conducted in PubMed and the Cochrane Library up to February 2024, and the search strategy was developed as follows: (43 conditions/diseases MeSH term and text word) AND (acupuncture MeSH term and text word) AND (systematic review MeSH term and text word). A detailed search strategy for each condition/disease and search results was suggested (Supplementary material 1).
3. Data extraction and analysis
To investigate how the status of evidence for symptoms (conditions/diseases) considered to have uncertain effects over the past 10 years has changed, and whether the reasons for insufficient evidence that served as a basis for these assessments has changed, data on the effectiveness of acupuncture, “literature size” (number of RCTs included in a SR), and level of confidence presented in the 2014 report [1] were extracted. However, since the evaluation of effectiveness analyses and various outcomes at different points in time (even within a single SR), only the primary outcome was assessed for effect. Confidence level were categorized according to the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) assessment as high, moderate, low, or very low and GRADE assessments in each SR was extracted for the included SRs [4]. Additionally, in cases where there was a debate on the evidence, reasons why the evidence could not be definitively assessed were identified and extracted from the respective reviews.
Results
A total of 43 conditions/diseases (7 for pain, 17 for wellness and 19 for mental health) were assessed in this 2024 reassessment meta-epidemiology study. When comparing the “literature size” in the 2014 report and in the 2024 reassessment, the average number of RCTs increased from 11 to 19.5 for the included conditions/diseases. In the RCTs of the 2014 report, the confidence level of evidence was generally high (n = 13) or moderate (n = 25), but it worsened in the 2024 reassessment. However, more conditions/diseases were identified to be effectively treated with acupuncture in the 2024 reassessment compared with those in the 2014 report [1] (Table 1).
Overview of Evidence Gap Between 2014 and 2024
1. Change in the level of confidence in RCTs between 2014 and 2024
For six conditions/diseases, there was no change in the level of confidence level between 2014 [1] and 2024. However, 17 conditions/diseases showed a worse level of evidence in 2024. Twenty SRs did not adopt the GRADE assessment so a change in the level of evidence could not be identified (Table 2 [5–84]). Limitations due to methodological issues, such as poor methodological quality in the RCTs, a small number of RCTs, and small sample sizes in each RCT were suggested as major reasons for the unclear effect or low confidence level in the 2014 report [1]. In addition, where benefits had been observed in some cases, effect sizes were small. For some conditions/diseases, inconsistency (conflicting results) was observed in results between individual SRs. Such inconsistency was pointed out as another reason, an obstacle to the firm conclusions in some conditions/diseases that acupuncture was effective (Supplementary material 2). Significant change was not observed in the 2024 reassessments. The certainty of evidence for conditions/diseases varies from very low to moderate. The reasons for this variability were related to factors such as a high risk of bias of the RCTs included in the study, particularly from the perspectives in blinding of participants and acupuncture practitioners, poor study design, and poor reporting. Furthermore, the small sample sizes in some RCTs contributed to inconsistency and imprecision in results (Supplementary data 3).
Level of Confidence Evaluation in the 2014 Report and in the 2024 Reassessment
Discussion
In this meta-epidemiology study, a reassessment was conducted (a decade later in 2024) of the acupuncture evidence on 43 conditions/diseases that were categorized as unclear or potentially effective in the "Evidence Map of Acupuncture" published in 2014 [1]. This re-evaluation aimed to ascertain any changes in the clinical evidence of acupuncture to explore if the current assessment would still regard the evidence as uncertain, identify the reasons for such uncertainty, and determine if there have been any shifts from the "Evidence Map of Acupuncture" 2014 [1] findings. The findings of this meta-epidemiology study revealed that the certainty of the evidence supporting the effect of acupuncture for most of the evaluated conditions/diseases remains low. Furthermore, it was observed that the factors contributing to this uncertainty have remained unchanged or even worsened since the 2014 report [1].
Contributing factors for a low level of certainty or unclear effect of acupuncture suggested in the 2014 report [1] included poor research methodology, small sample size and inconsistency of the results across the RCTs included in the study, and these are also pointed out as the main reasons for low certainty of evidence for acupuncture in 2024. Although the number of RCTs evaluating the evidence for individual conditions/diseases is increasing, there is a lack of evidence indicating that this numerical increase of clinical studies contributes to a change in the certainty of evidence. The certainty of acupuncture evidence is not related to a rise in the number of RCTs, and this leads to significant questions. This could be interpreted as casting a shadow over the hope that repeated RCTs on the same condition/disease, and the accumulation of their results, could establish certainty in the evidence supporting acupuncture. This situation underscores a critical challenge in enhancing the quality and impact of evidence in the field, suggesting that simply increasing the quantity of RCTs may not be sufficient to substantiate the effectiveness of acupuncture for various conditions/diseases.
In order to establish robust evidence on the effect of acupuncture, what should we do? Firstly, it is necessary to develop clinical research methodologies or a new critical appraisal tool that align with the characteristics of acupuncture treatment. Due to its intrinsic nature, acupuncture research presents significant challenges in achieving practitioner and participant blinding, a hallmark of rigor in clinical trials [85]. Unlike pharmaceutical interventions, invasive procedures or interventions like acupuncture often hinge on the practitioner's skill and technique for success. To date, no sham acupuncture device has been developed which permits the full expression of the practitioner's technical skill [86]. Recognizing the difficulty of blinding in such contexts, it is essential to contemplate evaluation methods that do not compromise the level of evidence, provided that other contributing biases are adequately controlled.
Secondly, there is a critical need to minimize unnecessary research and consolidate research resources to conduct large-scale clinical studies. The lack of sufficient power in acupuncture studies has been extensively critiqued. Much of the uncertainty surrounding evidence in acupuncture relates to small sample sizes; studies need to be powered [87]. Emulating large-scale studies performed in Germany, it is crucial to perform clinical research that calculates realistic and valid sample sizes, ensuring adequate participant numbers reflective of those calculations [88].
Thirdly, the role and conception of sham acupuncture warrant re-evaluation. A notable observation from evidence reviews on acupuncture is the fluctuating evidence for acupuncture effectiveness for certain conditions/diseases based on the control group intervention. Instances have shown observable effects when a no-treatment group served as the control, whereas significant effects were often not detected with the use of sham acupuncture as a control [89]. Despite the development of various types of sham acupuncture for blinding purposes, it is fundamentally different from the placebo used in drug trials and has occasionally shown significant effects for various conditions/diseases, including specific effects of sham acupuncture itself. For these reasons, comparisons with sham acupuncture may undervalue the effectiveness of acupuncture treatment or, worse, lead to false-negative outcomes [90]. Instead of using sham acupuncture, comparative trials assessing non-inferiority to standard treatments are alternative options for this issue.
Fourthly, considering the synthesis of evidence on acupuncture, it is essential to address the issues of heterogeneity in acupuncture interventions, and research design. Acupuncture treatments vary significantly in terms of therapeutic principles, acupoints selected, needle length and gauge, frequency, and total duration of treatment, as well as the method, insertion depth, and direction of stimulation [91]. Such clinical heterogeneity often acts as a barrier to providing clear evidence [92]. Furthermore, differences in treatment durations, and types and duration of outcome assessments in clinical trial design are also important limitations to consider in evidence synthesis [93]. To establish robust evidence for acupuncture, it is necessary to standardize research design and acupuncture methods to ensure that results from primary studies, such as RCTs, can be integrated into future SRs. Suggesting standard acupuncture treatment protocols based on international expert consensus for conditions/diseases with unclear evidence, and defining essential outcomes to be assessed in these conditions/diseases, will aid in mitigating the issue of heterogeneity in evidence synthesis when conducting future RCTs.
This meta-epidemiology study has limitations which should be declared here. As suggested, evidence mapping is comparatively a new type of research and there is no standard methodology when selecting literature and assessing certainty. One main reason for decreased level of certainty in the effects of acupuncture treatment was that the reassessment in 2024 used GRADE assessment results for the SRs which were not used in 2014 report. Despite such a significant limitation, we believe that our approach was appropriate in achieving the goal of illustrating the current situation of the evidence.
Looking to the future, it raises the question of whether we can reach a conclusive understanding of acupuncture effectiveness within the next decade. This issue requires profound consideration.
Supplementary Materials
Supplementary materials are available at doi: https://doi.org/10.56986/pim.2024.06.006.
Notes
Author Contributions
Conceptualization, writing - original draft, and writing - review & editing: THK and JWK.
Conflicts of Interest
There is no conflict of interest to declare.
Funding
None.
Ethical Statement
This article did not include any personal information and general research ethics guidelines were followed.
Data Availability
There is no usable data in this article.