Artificial Intelligence in the Editorial and Peer Review Process: A Protocol for a Cross-Sectional Survey of Traditional, Complementary, and Integrative Medicine Journal Editors’ Perceptions
Article information
Abstract
This research protocol outlines a cross-sectional survey study, aimed at editors from traditional, complementary, and integrative medicine (TCIM) journals, regarding their perceptions of the use of artificial intelligence (AI) in the editorial and peer review process. The survey will be sent to editors-in-chief, associate editors, and editorial board members of TCIM journals (100–150). The research involves purposive sampling via manual collection of contact details from TCIM journal websites. The survey will include sections on demographics, current use and familiarity with AI, perceived benefits and challenges, ethical concerns, and the outlook for AI in publishing. Data collection will be conducted online using SurveyMonkey whereby email invitations and follow-up reminders will be sent to potential respondents. Quantitative data will be analyzed using descriptive statistics, and for qualitative data, thematic analyses will be employed. This protocol study aimed to devise a survey which could provide insight into the acceptance and potential barriers to AI adoption in TCIM publishing from an editor’s perspective. The results of which may later guide the development of AI tools in a way that aligns with the needs and values of the TCIM research community.
Introduction
Artificial intelligence (AI) is increasingly being integrated into various stages of the publishing process, and offers promising tools to enhance efficiency, accuracy, and objectivity of the research process [1]. In the editorial and peer review process, AI applications range from automated manuscript screening and plagiarism detection to recommending reviewers, and providing initial reviews [2,3]. These technologies aim to streamline workflows, reduce the time to publication, and ensure the integrity of scientific literature.
Traditional, complementary, and integrative medicine (TCIM) is a diverse field encompassing various practices and treatments (such as acupuncture, herbal remedies, and meditative techniques) which differ, by cultural context, in countries that are not traditionally part of conventional medical care [4]. The editorial and peer review process in TCIM journals can be particularly challenging due to the interdisciplinary nature of the research, diversity of methodologies, and need for rigorous scientific evaluation (to maintain credibility) [5,6]. AI offers unique opportunities to address these challenges associated with the evaluation of complex methodologies and enhancement of study reproducibility, which in turn, can help establish and uphold high standards of evidence [2,7,8].
Despite the potential benefits, the adoption of AI in the editorial and peer review process within TCIM journals is not well-documented. There is a paucity of empirical data on how editors perceive the role of AI, its benefits, challenges, and ethical implications in this specific context [9,10]. Understanding these perceptions is crucial for informing the development and implementation of AI tools that are tailored to the needs of TCIM journals, and for addressing any concerns that may hinder their adoption [10]. This knowledge gap underscores the need for comprehensive research to evaluate the attitudes and experiences of TCIM journal editors regarding the role of AI in the peer review and editorial processes.
Exploring the perceptions of TCIM journal editors regarding AI is timely and significant. AI can potentially enhance efficiency by reducing the workload of editors and reviewers through the automation of routine tasks, and allow them to focus on more critical aspects of the review process [2,7]. In addition, AI tools can help improve the quality and consistency of manuscript evaluations, reduce human bias, and enhance the overall quality of published research [11,12]. Addressing ethical considerations, such as transparency, accountability, and potential bias associated with AI, is essential for gaining the trust of editors and the broader scientific community [1,12,13].
Insights on the current use and familiarity with AI in TCIM publishing, perceived benefits and challenges, ethical concerns, and the outlook for AI in publishing may guide the development of AI tools that are more effectively aligned with the specific needs and values of the TCIM research community.
In summary, this protocol aims to devise a survey to fill the existing knowledge gap by providing a comprehensive assessment of TCIM journal editors’ perceptions of AI in the editorial and peer review process. The findings of the survey may have important implications for the future integration of AI technologies in academic publishing, particularly within the field of TCIM.
Materials and Methods
1. Open science statement
This study will be registered and made available on the Open Science Framework (https://osf.io/) prior to any data collection. Relevant study materials and data, such as de-identified survey responses will also be uploaded to Open Science Framework, and a final version of the manuscript will be posted as a preprint prior to submission to a peer-reviewed journal.
2. Study design
The study design is an anonymous, online cross-sectional survey targeted at the editorial board members of journals focused on TCIM. It is estimated that between 100 and 150 TCIM journals will be included. The survey questions were adapted from a previous study led by JYN [14] (the survey for the present study can be found in the Supplementary Materials).
3. Sample and sampling method
The inclusion criteria for participants of this study are editors-in-chief, associate editors, and other editorial board members of TCIM journals that are directly involved in the peer review and/or editing processes of submitted manuscripts. TCIM journals will be identified following the search criteria detailed in Table 1. For each journal, the names of all editors and editorial board members will be collected from the journal webpage. The email addresses of these editors will be manually searched and extracted. Duplicate email addresses will be removed. Editors who are strictly involved with the manuscript formatting aspects of the editorial process (e.g., copyeditors, and technical editors), statistical editors, and other noneditorial staff that are not responsible for TCIM content contained within manuscripts submitted will be excluded.
Search Criteria to Identify Traditional, Complementary, and Integrative Medicine Journals
The survey will be closed; only invitees will have access to this survey, and they will be instructed not to share the survey link. Emails will be sent out to the prospective participants via SurveyMonkey. This initial email will include a description of the study, the objectives, and a link to access the survey. Clicking on the survey link will lead the participants to an informed consent form. After the participants provide consent, they will be taken to the survey questions. Numbers of invalid and nonfunctional emails which bounce back will be recorded by SurveyMonkey. Financial compensation will not be provided for participation. In addition, there is no requirement for those emailed a survey link to partake in the study.
4. Survey development
The survey will contain 38 consecutive questions, spanning nearly 8 pages, and it should take approximately 15 minutes for participants to complete. The survey was developed as a structured questionnaire that begins with a screening question for eligibility, and that features sections on demographics and background information, current use and familiarity with AI in editorial tasks, perceived benefits and challenges of AI, ethical concerns and potential biases of AI, and perceptions of the outlook on AI in publishing. The questionnaire primarily includes questions in a multiple-choice format, a matrix-style question asking participants to follow a Likert scale to describe their perceptions of the implementation of AI in editing and peer-reviewing, and an open-ended question. Participants will have the option to skip questions (with the exception of the screening question) they do not wish to answer, and no personal identifying information will be collected. The questionnaire will undergo pilot testing with a small group of editors. Their feedback will be incorporated, as appropriate, in an attempt to improve the face validity of the survey.
5. Data collection
Data will be collected via an online survey using SurveyMonkey (https://www.surveymonkey.com/). The survey will remain available for one month. Participants will be recruited through email invitations containing a link to the survey, and to increase response rates, 3 follow-up reminders will follow spaced 7 days apart.
6. Data analysis
Descriptive statistics (e.g., ratios, percentages) for demographic data will be generated from the analysis of the quantitative data. These statistics will be presented in a graph or chart format to visually summarize key demographic characteristics. Qualitative data from open-ended responses will be analyzed thematically. A data driven approach will serve to analyze the responses to the open-ended question. Open-ended responses will be analyzed using inductive coding and thematic analysis [15,16]. Each response will be assigned a code that captures the core meaning, which will then be grouped into broader themes based on emerging patterns and commonalities. Themes will be developed independently by multiple authors working in parallel to ensure reliability. An inductive approach, without reliance on pre-existing theories, will guide the analysis to remain grounded in the data [15,16]. To inform the researchers’ protocol design, and reporting of the survey, the Checklist for Reporting Results of Internet E-Surveys and Strengthening the Reporting of Observational Studies in Epidemiology will be used [17,18].
Discussion
The primary objective of this protocol was to design a survey study to assess the perceptions of editors of TCIM journals regarding the use of AI in the editorial and peer review process. By exploring these perceptions, the study aims to identify perceived benefits, challenges, and ethical considerations associated with AI in academic publishing [1,5,11]. The findings from the survey study may have significant implications for the future integration of AI technologies in the publishing workflow of TCIM journals. Understanding editors’ perceptions will provide valuable insight into the acceptance and potential barriers to AI adoption, and guide the development of AI tools that are aligned with the specific needs and values of the TCIM research community [2,9,10]. In addition, the survey study may contribute to the broader discussion on the ethical use of AI in scholarly publishing, highlighting areas where transparency and accountability need to be reinforced [1,19].
The strengths of this protocol for the survey study include its comprehensive approach to data collection, and the targeted sampling method. By using TCIM journals that will be identified using strict inclusion criteria, and manually collecting the names and email addresses of editors, this study protocol will ensure a robust and representative sample of the target population. The structured questionnaire, which will be validated through pilot testing, aims to facilitate the collection of detailed and relevant data on TCIM editors’ perceptions. However, the protocol for the study has inherent limitations including the reliance on self-reported data which may introduce response bias such as editors that are more interested in AI may be more likely to participate. In addition, the manual collection of email addresses, while thorough, is time-consuming, and may result in some editors being inadvertently missed. Despite these limitations, the protocol for the study may result in a survey which provides valuable insights into the current state of AI integration in the editorial and peer-review processes of TCIM journal publishing, and provide the basis for future research and technological advancements in this field.
Supplementary Materials
Supplementary materials are available at doi: https://doi.org/10.56986/pim.2025.06.008
Notes
Author Contributions
Designed and conceptualized the study, co-drafted the manuscript, and gave final approval of the version to be published: JYN. Co-drafted the manuscript, and gave final approval of the version to be published: DB. Co-drafted the manuscript, and gave final approval of the version to be published: ND. Made critical revisions to the manuscript, and gave final approval of the version to be published: MSL. Made critical revisions to the manuscript, and gave final approval of the version to be published: YL.
Conflicts of Interest
The authors declare that they have no competing interests.
Author Use of AI Tools Statement
AI tools were not involved in any part of the scientific research, including data interpretation or result formulation.
Funding
This writing of this protocol was unfunded.
Ethical Statement
Ethics approval for this study was granted by the University Hospital Tübingen Research Ethics Board prior to beginning this project (REB no.: 081/2025BO2).
Data Availability
All relevant data are included in this manuscript.