Does the Chinese Literature Indicate Larger Effect Sizes? This Might Indeed be the Case

Tae-Hun Kim; Jung Won Kang

doi:10.56986/pim.2023.10.009

Articles

Page Path: HOME > Perspect Integr Med > Volume 2(3); 2023 > Article

Letter Does the Chinese Literature Indicate Larger Effect Sizes? This Might Indeed be the Case: Tae-Hun Kim¹, Jung Won Kang²; Perspectives on Integrative Medicine 2023;2(3):202-203.
DOI: https://doi.org/10.56986/pim.2023.10.009
Published online: October 23, 2023

¹Korean Medicine Clinical Trial Center, Korean Medicine Hospital, Kyung Hee University, Seoul, Republic of Korea

²Department of Acupuncture & Moxibustion, College of Korean Medicine, Kyung Hee University, Seoul, Repulic of Korea

*Corresponding author: Tae-Hun Kim, Korean Medicine Clinical Trial Center, Korean Medicine Hospital, Kyung Hee University, Seoul, Republic of Korea, Email: rockandmineral@gmail.com

• Received: October 11, 2023 • Revised: October 16, 2023 • Accepted: October 16, 2023

This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

367 Views
19 Download

Full Article

Download PDF

Abstract
Acknowledgments
Article information
Data Availability
References

Abstract

This letter discusses concerns in a currently published meta-epidemiological study on commonly observed large effect sizes in the Chinese literature, focusing on potential selection bias and analysis methods. Researchers should be cautious when conducting systematic reviews that include the literature from specific countries or regions. Regardless of the country, the key issue is to enhance future research quality.
Keywords: Chinese literature, acupuncture, publication bias, large effect size

Dear editor,

We have recently read with interest, a meta-epidemiological study on effect sizes in Chinese literature, published in BMC Medical Research Methodology [1]. The issue has long been the subject of debate, addressed by the authors, and has been recognized by scholars as having empirical evidence [2]. Having been involved in complementary and alternative medicine research for several years and having encountered a significant amount of Chinese literature, we have always been mindful of these concerns. We are grateful that this meta-epidemiological study revisited this issue and provided conclusions that support our long-standing observations on this matter. According to the authors’ findings, acupuncture trials reported in Chinese showed significantly larger effect sizes than those in non-Chinese languages (ratio odds ratio 0.51, 95% confidence intervals 0.29–0.91) and they concluded that high risk of bias and study population are significant factors affecting the large effect size [1]. We would like to address and discuss some of these points.

First, the research methodology of this study needs to be revisited. Rather than directly searching for original trials to be included in the analysis, the authors located individual studies from Cochrane reviews, which may have inadvertently introduced selection bias. Depending on author composition and research objectives, Chinese literature may not always be included in Cochrane reviews [3]. According to the authors, only 37% of the 84 systematic reviews included in this study were found in Chinese databases. Thus, there is a considerable possibility of limiting the number of Chinese publications.

Second, we would like to mention the method used to analyze differences between comparison groups. The authors calculated ratio odds ratios to examine differences between articles published in Chinese and non-Chinese languages, or between Chinese and other populations. However, it might have been more appropriate to analyze the impact of publication language and study population on effect size using meta-regression.

Third, the reasoning process leading to the conclusion seems to have some issues. The authors attempted to demonstrate that the language of the trial (Chinese or non-Chinese) is not an influential factor in explaining large effect sizes by comparing studies with a high or unclear risk of bias. However, estimating the actual impact through a comparative analysis of all studies and presenting comparisons between high-risk and low-risk subgroups may provide greater insight. Similarly, in the context of comparing Chinese and non-Chinese populations, a fair interpretation can solely be achieved through the concurrent presentation of subgroup results for studies involving both high and low risk of bias. Using the methodology employed in this study to analyze data could result in a potentially biased perspective. For instance, this methodology does not completely address Chinese-language publications with significant effect sizes and with a low risk of bias. Furthermore, the correlation between Chinese-language publications and effect size in studies of the Chinese population is not covered in the illustrated method. Evidently, issues related to the research question are inadequately covered by the current method.

In conclusion, it is critical to exercise caution when conducting a comprehensive review of the literature, including publications from China. Whether in a positive or negative way, the volume of clinical evidence on acupuncture is significantly influenced by studies conducted and reported from China. Continual endeavors to enhance the caliber of research are imperative, irrespective of its location.

Acknowledgments

BMC Medical Research Methodology stated that this letter could not be published in its journal. Thus, we submitted this article to Perspectives on Integrative Medicine.

Article information

Author Contributions

Conceptualization: THK and JWK. Writing - review and editing: THK and JWK.

Conflicts of Interest

There is no conflict of interest.

Funding

None.

Ethical Statement

This letter did not include any personal information. We followed general research ethics guidelines for this letter.

Data Availability

There is no usable data in this article.

References

[1] Li J, Hui X, Yao L, Shi A, Yan P, Yao Y, et al. The relationship of publication language, study population, risk of bias, and treatment effects in acupuncture related systematic reviews: a meta-epidemiologic study. BMC Med Res Methodol 2023;23(1):96. Article PubMed PMC PDF
[2] Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials 1998;19(2):159−66.Article PubMed
[3] Cohen JF, Korevaar DA, Wang J, Spijker R, Bossuyt PM. Should we search Chinese biomedical databases when performing systematic reviews? Syst Rev 2015;4:23. Article PubMed PMC PDF

Figure & Data

References

Citations

Citations to this article as recorded by

PubReader
ePub Link

Cite

Download Citation

Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.

Format:

RIS — For EndNote, ProCite, RefWorks, and most other reference management software
BibTeX — For JabRef, BibDesk, and other BibTeX-specific software

Include:

Citation for the content below
Citation and abstract for the content below

Does the Chinese Literature Indicate Larger Effect Sizes? This Might Indeed be the Case

Perspect Integr Med. 2023;2(3):202-203. Published online October 23, 2023

DOI: https://doi.org/10.56986/pim.2023.10.009

XML Download