research article for journal publication

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts

Research articles

DNA glycosylases provide antiviral defence in prokaryotes

A screen utilizing an environmental DNA library in Escherichia coli is used to identify Brig1, a previously unknown anti-phage defence system with homologues across distinct clades of bacteria.

• Amer A. Hossain
• Ying Z. Pigli
• Luciano A. Marraffini

Hybrid speciation driven by multilocus introgression of ecological traits

Genomic studies of Heliconius butterflies provide evidence that Heliconius elevatus is a hybrid species, and that its speciation was driven by introgression of traits from Heliconius melpomene into the other parent, an ancestor of Heliconius pardalinus .

• Neil Rosser
• Fernando Seixas
• Kanchon K. Dasmahapatra

Methane emission from a cool brown dwarf

Methane emission from a very cool brown dwarf, perhaps arising from an aurora, has been detected in James Webb Space Telescope observations.

• Jacqueline K. Faherty
• Ben Burningham
• Niall Whiteford

Stepwise activation of a metabotropic glutamate receptor

We propose a model for a sequential, multistep activation mechanism of metabotropic glutamate receptor subtype 5, including a series of structures in lipid nanodiscs, from inactive to fully active, with agonist-bound intermediate states.

• Kaavya Krishna Kumar
• Haoqing Wang
• Brian K. Kobilka

The economic commitment of climate change

Analysis of projected sub-national damages from temperature and precipitation show an income reduction of 19% of the world economy within the next 26 years independent of future emission choices.

• Maximilian Kotz
• Anders Levermann
• Leonie Wenz

Neural crest origin of sympathetic neurons at the dawn of vertebrates

Challenging the belief that sympathetic ganglia are an innovation of jawed vertebrates, a study reports the presence of sympathetic neurons in an extant jawless vertebrate, the sea lamprey Petromyzon marinus .

• Brittany M. Edens
• Marianne E. Bronner

Refining the impact of genetic evidence on clinical success

Human genetic evidence increases the success rate of drugs from clinical development to approval but we are still far from reaching peak genetic insights to aid the discovery of targets for more effective drugs.

• Eric Vallabh Minikel
• Jeffery L. Painter
• Matthew R. Nelson

Seismological evidence for a multifault network at the subduction interface

Using observations of double-difference relocated earthquakes in a local three-dimensional velocity model for Ecuador, a detailed image of seismicity is created, forming the base for more realistic models of earthquake rupture, slip and hazard in subduction zones.

• Caroline Chalumeau
• Hans Agurto-Detzel
• Audrey Galve

Biogeographic response of marine plankton to Cenozoic environmental changes

Analysis of a global dataset reveals spatiotemporal patterns of marine plankton and their biogeographical responses during climatic and environmental changes across the Cenozoic era.

• Anshuman Swain
• Adam Woodhouse
• Christopher M. Lowery

Control of working memory by phase–amplitude coupling of human hippocampal neurons

Hippocampal theta–gamma phase–amplitude coupling integrates cognitive control and working memory storage across brain areas in humans.

• Jonathan Daume
• Jan Kamiński
• Ueli Rutishauser

Network-level encoding of local neurotransmitters in cortical astrocytes

A study investigates subcellular, single-cell and network-level comunication within the astrocyte network in response to the two major neurotransmitter inputs.

• Michelle K. Cahill
• Max Collard
• Kira E. Poskanzer

Corner- and edge-mode enhancement of near-field radiative heat transfer

Near-field radiative heat transfer between two coplanar silicon carbide membranes in close proximity is enhanced by the electromagnetic corner and edge modes.

• Lívia M. Corrêa
• Chris Dames

Brain endothelial GSDMD activation mediates inflammatory BBB breakdown

Lipopolysaccharide-induced breakdown of the blood–brain barrier requires activation of GSDMD-mediated plasma membrane permeabilization and pyroptosis in brain endothelial cells.

Stripped-envelope supernova light curves argue for central engine activity

Analysis of the energy budget of a sample of 54 well-observed stripped-envelope supernovae of all sub-types shows statistically significant, largely model-independent, observational evidence for a non-radioactive power source in most of them.

• Ósmar Rodríguez

Environmental drivers of increased ecosystem respiration in a warming tundra

Datasets from in situ warming experiments across 28 arctic and alpine tundra sites covering  a span of less than 1 year up to 25 years show the importance of local soil conditions and warming-induced changes therein for future climatic impacts on ecosystem respiration.  

• J. Dietrich
• E. Dorrepaal

Machine learning reveals the control mechanics of an insect wing hinge

Measurements of fly muscle activity using a genetically encoded calcium indicator and high-speed imaging of wing movement were used to construct a model of the insect wing hinge and the role of steering muscles in flight control.

• Johan M. Melis
• Igor Siwanowicz
• Michael H. Dickinson

Digital colloid-enhanced Raman spectroscopy by single-molecule counting

Research published in Nature shows that surface-enhanced Raman spectroscopy carried out with colloids can quantify a range of molecules down to concentrations at the femtomolar level.

• Daniel M. Czajkowsky

Streptomyces umbrella toxin particles block hyphal growth of competing species

Streptomyces are discovered to produce antibacterial protein complexes that selectively inhibit the hyphal growth of related species, a function distinct from that of the small-molecule antibiotics they are known for.

• Qinqin Zhao
• Savannah Bertolli
• Joseph D. Mougous

Laser spectroscopy of triply charged 229 Th isomer for a nuclear clock

The trapping of triply charged 229m Th 3+ is described and its nuclear decay half-life determined, showing useful properties for the development of a nuclear clock and applications in the search for new physics.

• Atsushi Yamaguchi
• Yudai Shigekawa
• Hidetoshi Katori

Promiscuous G-protein activation by the calcium-sensing receptor

Structures of the human calcium-sensing receptor can be bound into complex with G proteins from three different Gα subtypes while maintaining G-protein-binding specificity.

• Jinseo Park
• Qing R. Fan

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

• PLOS Biology
• PLOS Climate
• PLOS Complex Systems
• PLOS Computational Biology
• PLOS Digital Health
• PLOS Genetics
• PLOS Global Public Health
• PLOS Medicine
• PLOS Mental Health
• PLOS Neglected Tropical Diseases
• PLOS Pathogens
• PLOS Sustainability and Transformation
• PLOS Collections

Understanding the Publishing Process

What’s happening with my paper? The publication process explained

The path to publication can be unsettling when you’re unsure what’s happening with your paper. Learn about staple journal workflows to see the detailed steps required for ensuring a rigorous and ethical publication.

Your team has prepared the paper, written a cover letter and completed the submission form. From here, it can sometimes feel like a waiting game while the journal has your paper.  It can be unclear exactly who is currently handling your paper as most individuals are only involved in a few steps of the overall process. Journals are responsible for overseeing the peer review, publication and archival process: editors, reviewers, technical editors, production staff and other internal staff all have their roles in ensuring submissions meet rigorous scientific and ethical reporting standards. 

Read on for an inside look at how a conventional peer-reviewed journal helps authors transform their initial submission to a certified publication. 

Note that the description below is based on the process at PLOS journals. It is likely that at other journals, various roles (e.g. technical editor) may in fact also be played by the editor, and some journals may not have journal staff at all, with all roles played by volunteer academics. As such, please consider the processes and waypoints, rather than who performs them, as the key information.

Internal Checks on New Submissions

Estimated time: 10 days.

When a journal first receives your submission, there are typically two separate checks to confirm that the paper is appropriate and ready for peer review:

• Technical check.   Performed by a technical editor to ensure that the submission has been properly completed and is ready for further assessment. Blurry figures, missing ethical statements, and incomplete author affiliations are common issues that are addressed at this initial stage. Typically, there are three technical checks: upon initial submission, alongside the first decision letter, and upon acceptance. 
• Editorial screening . Once a paper passes the first check, an editor with subject expertise assesses the paper and determines whether it is within the journal’s scope and if it could potentially meet the required publication criteria. While there may be requests for further information and minor edits from the author as needed, the paper will either be desk rejected by the editor or allowed to proceed to peer review. 

Both editors at this point will additionally make notes for items to be followed-up on at later stages. The publication process involves finding a careful balance for when each check occurs. Early checks need to be thorough so that editors with relevant expertise can focus on the scientific content and more advanced reporting standards, but no one wants to be asked to reformat references only to have their paper desk rejected a few days later. 

Peer Review

Estimated time: 1 month.

Depending on the journal’s editorial structure, the editor who performed the initial assessment may also oversee peer review or another editor with more specific expertise may be assigned.  Regardless of the journal’s specific process, the various roles and responsibilities during peer review include:  

When you have questions or are unsure who your manuscripts is currently with, reach out to the journal staff for help (eg. [email protected]). They will be your lifeline, connecting you to all the other contributors working to assess the manuscript. 

Whether an editor needs a reminder that all reviews are complete or a reviewer has asked for an extension, the journal acts as a central hub of communication for those involved with the publication process. As editors and reviewers are used to hearing from journal staff about their duties, any messages you send to the journal can be forwarded to them with proper context and instructions on how to proceed appropriately. Additionally, journal staff will be able to inform you of any delays, such as reviewer availability during summer and holiday periods. 

Revision Decision

Estimated time: 1 day.

Editors evaluate peer reviewer feedback and their own expert assessment of the manuscript to reach a decision. After your editor submits a decision on your manuscript, the journal may review it before formally processing the decision and sending it on to you. 

A technical editor may scan the manuscript and the review comments to ensure that journal standards have been followed. At this stage, the technical editor will also add requests to ensure the paper, if published, will adhere to journal requirements for data sharing, copyright, ethical reporting and the like. 

Performing the second technical check at this stage and adding the journal requirements to the decision letter ultimately saves time by allowing authors to resolve the journal’s queries while making revisions based on comments from the reviewers. 

Revised Submission Received

Estimated time: 3 days.

Upon receiving your revised submission, a technical editor will assess the revisions to confirm that the requests from the journal have been properly addressed. Before the paper is returned to the editor for their consideration, the journal needs to be confident that the paper won’t have any issues related to the metadata and reporting standards that could prevent publication. The editor may contact you to resolve any serious issues, though minor items can wait until the paper is accepted.

Subsequent Peer Review

Estimated time: 2 weeks, highly variable.

When your resubmitted paper has passed the required checks, it’ll be assigned back to the same editor who handled it during the first round of peer review. At this point, your paper has gone through two sets of journal checks and one round of peer review. If all has gone well so far, the paper should feel quite solid both in terms of scientific content and proper reporting standards. 

When the editor receives your revised paper, they are asked to check if all reviewer comments have been adequately addressed and if the paper now adheres to the journal’s publication criteria. Depending on the situation, some editors may feel confident making this decision based on their own expertise while others may re-invite the previous reviewers for their opinions. 

Individual responsibilities are the same as the initial round of peer review, but it is generally expected that later stages of peer review proceed quicker unless new concerns have been introduced as part of the revision. 

Preliminary Acceptance

Estimated time: 1 week.

Your editor is satisfied with the scientific quality of your work and has chosen to accept it in principle. Before it can proceed to production and typesetting, the journal office will perform it’s third and final technical check, requesting any formatting changes or additional details that may be required. 

When fulfilling these final journal requests, double check the final files to confirm all information is correct. If you need to make changes beyond those specifically required in the decision letter, inform the journal and explain why you made the unrequested changes. Any change that could affect the scientific meaning of the work will need to be approved by the handling editor. While including your rationale for the changes will help avoid delays, if there are extensive changes made at this point the paper may need to go through another round of formal review.

Formal Acceptance and Publication

Estimated time: 2 weeks.

After a technical editor has confirmed that all requests from the provisional acceptance letter have been addressed, you will receive your formal acceptance letter. This letter indicates that your paper is being passed from the Editorial department to the production department—that all information has been editorially approved. The scientific content has been approved through peer review, and the journal’s publication requirements have been met. 

Congratulations to you and your co-authors! Your article will be available as soon as the journal transforms the submission into a typeset, consistently structured scientific manuscript, ready to be read and cited by your peers.

The contents of the Peer Review Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

The contents of the Writing Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

There’s a lot to consider when deciding where to submit your work. Learn how to choose a journal that will help your study reach its audience, while reflecting your values as a researcher…

A Step-To-Step Guide to Write a Quality Research Article

• Conference paper
• First Online: 01 June 2023
• Cite this conference paper

• Amit Kumar Tyagi   ORCID: orcid.org/0000-0003-2657-8700 14 ,
• Rohit Bansal 15 ,
• Anshu 16 &
• Sathian Dananjayan   ORCID: orcid.org/0000-0002-6103-7267 17  

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 717))

Included in the following conference series:

• International Conference on Intelligent Systems Design and Applications

230 Accesses

19 Citations

Today publishing articles is a trend around the world almost in each university. Millions of research articles are published in thousands of journals annually throughout many streams/sectors such as medical, engineering, science, etc. But few researchers follow the proper and fundamental criteria to write a quality research article. Many published articles over the web become just irrelevant information with duplicate information, which is a waste of available resources. This is because many authors/researchers do not know/do not follow the correct approach for writing a valid/influential paper. So, keeping such issues for new researchers or exiting researchers in many sectors, we feel motivated to write an article and present some systematic work/approach that can help researchers produce a quality research article. Also, the authors can publish their work in international conferences like CVPR, ICML, NeurIPS, etc., or international journals with high factors or a white paper. Publishing good articles improve the profile of researchers around the world, and further future researchers can refer their work in their work as references to proceed with the respective research to a certain level. Hence, this article will provide sufficient information for researchers to write a simple, effective/impressive and qualitative research article on their area of interest.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Nair, M.M., Tyagi, A.K., Sreenath, N.: The future with industry 4.0 at the core of society 5.0: open issues, future opportunities and challenges. In: 2021 International Conference on Computer Communication and Informatics (ICCCI), pp. 1–7 (2021). https://doi.org/10.1109/ICCCI50826.2021.9402498

Tyagi, A.K., Fernandez, T.F., Mishra, S., Kumari, S.: Intelligent Automation Systems at the Core of Industry 4.0. In: Abraham, A., Piuri, V., Gandhi, N., Siarry, P., Kaklauskas, A., Madureira, A. (eds.) ISDA 2020. AISC, vol. 1351, pp. 1–18. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-71187-0_1

Chapter   Google Scholar  

Goyal, D., Tyagi, A.: A Look at Top 35 Problems in the Computer Science Field for the Next Decade. CRC Press, Boca Raton (2020) https://doi.org/10.1201/9781003052098-40

Tyagi, A.K., Meenu, G., Aswathy, S.U., Chetanya, V.: Healthcare Solutions for Smart Era: An Useful Explanation from User’s Perspective. In the Book “Recent Trends in Blockchain for Information Systems Security and Privacy”. CRC Press, Boca Raton (2021)

Google Scholar  

Varsha, R., Nair, S.M., Tyagi, A.K., Aswathy, S.U., RadhaKrishnan, R.: The future with advanced analytics: a sequential analysis of the disruptive technology’s scope. In: Abraham, A., Hanne, T., Castillo, O., Gandhi, N., Nogueira Rios, T., Hong, T.-P. (eds.) HIS 2020. AISC, vol. 1375, pp. 565–579. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73050-5_56

Tyagi, A.K., Nair, M.M., Niladhuri, S., Abraham, A.: Security, privacy research issues in various computing platforms: a survey and the road ahead. J. Inf. Assur. Secur. 15 (1), 1–16 (2020)

Madhav, A.V.S., Tyagi, A.K.: The world with future technologies (Post-COVID-19): open issues, challenges, and the road ahead. In: Tyagi, A.K., Abraham, A., Kaklauskas, A. (eds.) Intelligent Interactive Multimedia Systems for e-Healthcare Applications, pp. 411–452. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-6542-4_22

Mishra, S., Tyagi, A.K.: The role of machine learning techniques in the Internet of Things-based cloud applications. In: Pal, S., De, D., Buyya, R. (eds.) Artificial Intelligence-Based Internet of Things Systems. Internet of Things (Technology, Communications and Computing). Springer, Cham. https://doi.org/10.1007/978-3-030-87059-1_4

Pramod, A., Naicker, H.S., Tyagi, A.K.: Machine Learning and Deep Learning: Open Issues and Future Research Directions for Next Ten Years. Computational Analysis and Understanding of Deep Learning for Medical Care: Principles, Methods, and Applications. Wiley Scrivener (2020)

Kumari, S., Tyagi, A.K., Aswathy, S.U.: The Future of Edge Computing with Blockchain Technology: Possibility of Threats, Opportunities and Challenges. In the Book Recent Trends in Blockchain for Information Systems Security and Privacy. CRC Press, Boca Raton (2021)

Dananjayan, S., Tang, Y., Zhuang, J., Hou, C., Luo, S.: Assessment of state-of-the-art deep learning based citrus disease detection techniques using annotated optical leaf images. Comput. Electron. Agric. 193 (7), 106658 (2022). https://doi.org/10.1016/j.compag.2021.106658

Nair, M.M., Tyagi, A.K.: Privacy: History, Statistics, Policy, Laws, Preservation and Threat analysis. J. Inf. Assur. Secur. 16 (1), 24–34 (2021)

Tyagi, A.K., Sreenath, N.: A comparative study on privacy preserving techniques for location based services. Br. J. Math. Comput. Sci. 10 (4), 1–25 (2015). ISSN: 2231–0851

Rekha, G., Tyagi, A.K., Krishna Reddy, V.: A wide scale classification of class imbalance problem and its solutions: a systematic literature review. J. Comput. Sci. 15 (7), 886–929 (2019). ISSN Print: 1549–3636

Kanuru, L., Tyagi, A.K., A, S.U., Fernandez, T.F., Sreenath, N., Mishra, S.: Prediction of pesticides and fertilisers using machine learning and Internet of Things. In: 2021 International Conference on Computer Communication and Informatics (ICCCI), pp. 1–6 (2021). https://doi.org/10.1109/ICCCI50826.2021.9402536

Ambildhuke, G.M., Rekha, G., Tyagi, A.K.: Performance analysis of undersampling approaches for solving customer churn prediction. In: Goyal, D., Gupta, A.K., Piuri, V., Ganzha, M., Paprzycki, M. (eds.) Proceedings of the Second International Conference on Information Management and Machine Intelligence. LNNS, vol. 166, pp. 341–347. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-9689-6_37

Sathian, D.: ABC algorithm-based trustworthy energy-efficient MIMO routing protocol. Int. J. Commun. Syst. 32 , e4166 (2019). https://doi.org/10.1002/dac.4166

Varsha, R., et al.: Deep learning based blockchain solution for preserving privacy in future vehicles. Int. J. Hybrid Intell. Syst. 16 (4), 223–236 (2020)

Tyagi, A.K., Aswathy, S U.: Autonomous Intelligent Vehicles (AIV): research statements, open issues, challenges and road for future. Int. J. Intell. Netw. 2 , 83–102 (2021). ISSN 2666–6030. https://doi.org/10.1016/j.ijin.2021.07.002

Tyagi, A.K., Sreenath, N.: Cyber physical systems: analyses, challenges and possible solutions. Internet Things Cyber-Phys. Syst. 1 , 22–33 (2021). ISSN 2667–3452, https://doi.org/10.1016/j.iotcps.2021.12.002

Tyagi, A.K., Aghila, G.: A wide scale survey on botnet. Int. J. Comput. Appl. 34 (9), 9–22 (2011). (ISSN: 0975–8887)

Tyagi, A.K., Fernandez, T.F., Aswathy, S.U.: Blockchain and aadhaar based electronic voting system. In: 2020 4th International Conference on Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, pp. 498–504 (2020). https://doi.org/10.1109/ICECA49313.2020.9297655

Kumari, S., Muthulakshmi, P.: Transformative effects of big data on advanced data analytics: open issues and critical challenges. J. Comput. Sci. 18 (6), 463–479 (2022). https://doi.org/10.3844/jcssp.2022.463.479

Article   Google Scholar  

Download references

Acknowledgement

We want to think of the anonymous reviewer and our colleagues who helped us to complete this work.

Author information

Authors and affiliations.

Department of Fashion Technology, National Institute of Fashion Technology, New Delhi, India

Amit Kumar Tyagi

Department of Management Studies, Vaish College of Engineering, Rohtak, India

Rohit Bansal

Faculty of Management and Commerce (FOMC), Baba Mastnath University, Asthal Bohar, Rohtak, India

School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, Tamilnadu, 600127, India

Sathian Dananjayan

You can also search for this author in PubMed   Google Scholar

Contributions

Amit Kumar Tyagi & Sathian Dananjayan have drafted and approved this manuscript for final publication.

Corresponding author

Correspondence to Amit Kumar Tyagi .

Editor information

Editors and affiliations.

Faculty of Computing and Data Science, FLAME University, Pune, Maharashtra, India

Ajith Abraham

Center for Smart Computing Continuum, Burgenland, Austria

Sabri Pllana

University of Bari, Bari, Italy

Gabriella Casalino

University of Jinan, Jinan, Shandong, China

Department of Computer Science and Engineering, Thapar Institute of Engineering and Technology, Patiala, Punjab, India

Ethics declarations

Conflict of interest.

The author declares that no conflict exists regarding the publication of this paper.

Scope of the Work

As the author belongs to the computer science stream, so he has tried to cover up this article for all streams, but the maximum example used in situations, languages, datasets etc., are with respect to computer science-related disciplines only. This work can be used as a reference for writing good quality papers for international conferences journals.

Disclaimer. Links and papers provided in the work are only given as examples. To leave any citation or link is not intentional.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Cite this paper.

Tyagi, A.K., Bansal, R., Anshu, Dananjayan, S. (2023). A Step-To-Step Guide to Write a Quality Research Article. In: Abraham, A., Pllana, S., Casalino, G., Ma, K., Bajaj, A. (eds) Intelligent Systems Design and Applications. ISDA 2022. Lecture Notes in Networks and Systems, vol 717. Springer, Cham. https://doi.org/10.1007/978-3-031-35510-3_36

Download citation

DOI : https://doi.org/10.1007/978-3-031-35510-3_36

Published : 01 June 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-35509-7

Online ISBN : 978-3-031-35510-3

eBook Packages : Intelligent Technologies and Robotics Intelligent Technologies and Robotics (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

How to write a research article to submit for publication

Pharmacists and healthcare professionals who are undertaking research should have an understanding of how to produce a research article for publication, and be aware of the important considerations relating to submission to a peer-reviewed journal.

Viennaslide / Alamy Stock Photo

Undertaking and performing scientific, clinical and practice-based research is only the beginning of the scholarship of discovery [1] . For the full impact of any research to be achieved and to have an effect on the wider research and scientific community, it must be published in an outlet accessible to relevant professionals [2] .

Scientific research is often published in peer-reviewed journals. Peer review is defined as the unbiased, independent, critical assessment of scholarly or research manuscripts submitted to journals by experts or opinion leaders [3] . The process and requirements of reviewers has been covered recently [4] . On account of this rigorous process, peer-reviewed scientific journals are considered the primary source of new information that impacts and advances clinical decision-making and practice [5] , [6] .

The development of a research article can be helpful for the promotion of scientific thinking [7] , [8] and the advancement of effective writing skills, allowing the authors to participate in broader scientific discussions that lie beyond their scope of practice or discipline [2] .

This article aims to provide pharmacists and healthcare professionals who are undertaking research with an understanding of how to produce a research article for publication, as well as points to consider before submission to a peer-reviewed journal.

Importance of the research question

This article will not go into detail about forming suitable research questions, however, in principle, a good research question will be specific, novel and of relevance to the scientific community (e.g. pharmacy – pharmacists, pharmaceutical scientists, pharmacy technicians and related healthcare professionals). Hulley et al . suggest using the FINER criteria (see ‘Figure 1: FINER criteria for a good research question’) to aid the development of a good research question [9] .

Figure 1: FINER criteria for a good research question

Source: Hulley S, Cummings S, Browner W  et al . [9]

The FINER criteria highlight useful points that may generally increase the chances of developing a successful research project. A good research question should specify the population of interest, be of interest to the scientific community and potentially to the public, have clinical relevance and further current knowledge in the field.

Having a clear research question that is of interest to those working in the same field will help in the preparation of an article because it can be used as the central organising principle – all of the content included and discussed should focus on answering this question.

Preparing a first draft

Before writing the article, it is useful to highlight several journals that you could submit the final article to. It also helps to familiarise yourself with these journals’ styles, article structures and formatting instructions before starting to write. Many journals also have criteria that research articles should be able to satisfy. For example, all research article submissions to  Clinical Pharmacist must demonstrate innovative or novel results that are sustainable, reproducible and transferable [10] .

Having researched potential target journals, you should have a clear idea about your target audience, enabling you to pitch the level of the article appropriately [11] (see ‘Box 1: Top tips to prepare for writing’).

Box 1: Top tips to prepare for writing

• Know the focus of the paper – identify two or three important findings and make these the central theme of the article;
• Gather important data, perform any analyses and make rough data plots and tables beforehand. These can then be refined for inclusion or submitted as supplementary information if needed;
• Organise your results to flow in a logical sequence;
• Know the structure and requirements of your target journals (check websites and author guidelines, as well as published articles);
• Think about the style of the piece and look to pitch the article at the level of the intended audience;
• Clarity should be your guiding principle.

Structuring a research article

Most research articles follow a similar structure and format that includes an abstract, introduction, methods, results and discussion, as well as a summary of the key points discussed in the article.

One approach is to start with the methods section, which can be derived from the protocol and any pilot phase. Many of the figures and tables can be constructed in advance, which will help with writing the results section. The questions addressed by the study can be used alongside the results to formulate the introduction, which can guide how the discussion is written [11] .

Clinical Pharmacist , like other peer-reviewed journals, has specific author guidelines and formatting instructions to help authors prepare their articles [10] , [12] , [13] . The following sections will discuss the required sections and important considerations for authors when writing.

Title, abstract and keywords

The title, abstract and keywords are essential to the successful communication of research. Most electronic search engines, databases (e.g. PubMed/MEDLINE) and journal websites extract words from them to determine whether your article will be displayed to interested readers [14] , [15] , [16] , [17] , enabling accurate dissemination and leading to future citations.

In addition, the title and abstract are usually freely available online. If an article is not published in an ‘open access’ format, (i.e. it is free and immediately available online and access is combined with the rights to use these articles fully in the digital environment) [18] , or if the reader does not have a subscription to the journal, they will have to decide on whether to pay to access the full article to continue reading. Therefore, it is imperative that they are informative and accurate.

The title should accurately reflect the research, identify the main issue and begin with the subject matter, while being both simple and enticing enough to attract the audience [19] . Authors should avoid using ‘a study of’, ‘investigations into’ and ‘observations on’ in titles. It is also worth remembering that abstracting and indexing services, such as MEDLINE, require accurate titles, because they extract keywords from them for cross-referencing [19] .

Many journals require the abstract to be structured in the same way as the main headings of the paper (e.g. introduction, methods, results, discussion and conclusion) and to be around 150–300 words in length [10] . In general, references should not be cited in the abstract.

Introduction

The introduction should provide the background and context to the study. Two or three paragraphs can be dedicated to the discussion of any previous work and identification of gaps in current knowledge. The rest of the introduction should then outline what this piece of work aims to address and why this is important, before stating the objectives of the study and the research question [20] .

The methods section should provide the reader with enough detail for them to be able to reproduce the study if desired [3] . The context and setting of the study should be described and the study design specified. The section should further describe the population (including the inclusion and exclusion criteria), sampling strategy and the interventions performed. The main study variables should be identified and the data collection procedures described [3] .

Authors should provide specific, technical and detailed information in this section. Several checklists and guidelines are available for the reporting of specific types of studies:

• CONSORT is used for developing and reporting a randomised controlled trial [21] ;
• The STARD checklist can help with designing a diagnostic accuracy study [22] ;
• The PRISMA checklist can be used when performing a metaâ€analyses or systematic review, but can also help with compiling an introduction [23] .

For the reporting of qualitative research studies, authors should explain which research tradition the study utilises and link the choice of methodological strategies with the research goals [24] .

For studies describing the development of new initiatives or clinical services, authors should describe the situation before the initiative began, the establishment of priorities, formulation of objectives and strategies, mobilisation of resources, and processes used in the methods section [10] .

The final portion of the methods section will include the statistical methods used to analyse the data [25] . The statistical methods employed should be described with enough detail to enable a knowledgeable reader with access to the original data to be able to judge its appropriateness for the study and verify the results [3] . For survey-based studies and information on sampling frame, size and statistical powers, see ‘When to use a survey in pharmacy practice research’ [26] .

Findings should be quantified and presented with appropriate indicators of measurement error or uncertainty (e.g. confidence intervals). Authors should avoid relying solely on statistical hypothesis testing, such as P values, because these fail to convey important information about effect size and precision of estimates [3] . Statistical terms, abbreviations and most symbols should be defined, and the statistical software package and versions used should be specified. Authors should also take care to distinguish prespecified from exploratory analyses, including subgroup analyses [3] .

The results section should be straightforward and factual and all of the results that relate to the research question should be provided, with detail including simple counts and percentages [27] . Data collection and recruitment should be commented on and the participants described. Secondary findings and the results of subgroup analyses can also be presented [27] .

Figures, schemes and tables

To present data and results of the research study, figures, schemes and tables can be used. They should include significant digits, error bars and levels of statistical significance.

Tables should be presented with a summary title, followed by caption, a sentence or two that describes the content and impact of the data included in the table. All captions should provide enough detail so that the table or figure can be interpreted and understood as stand-alone material, separate from the article.

Figures should also be presented with a summary title, a caption that describes the significant result or interpretation that can be made from the figure, the number of repetitions within the experiment, as well as what the data point actually represents. All figures and tables should be cited in the manuscript text [11] .

When compiling tables and figures, important statistics, such as the number of samples (n), the index of dispersion (standard deviation [SD], standard error of the mean [SEM]), and the index of central tendency (mean, median or mode), must be stated. The statistical analysis performed should also be included and specific statistical data should be indicated (e.g. P values) [11] .

Discussion and conclusions

The discussion section should state the main findings of the study. The main results should be compared with reference to previous research and current knowledge, and where this has been extended it should be fully described [2] , [11] , [25] . For clinical studies, relevant discussion of the implications the results may have on policy should be included [10] . It is important to include an analysis of the strengths and limitations of the study and offer perspectives for future work [2] . Excessive presentation of data and results without any discussion should be avoided and it is not necessary to cite a published work for each argument presented. Any conclusions should include the major findings, followed by a brief discussion of future perspectives and the application of this work to other disciplines [10] .

The list of references should be appropriate; important statements presented as facts should be referenced, as well as the methods and instruments used. Reference lists for research articles, however, unlike comprehensive reviews of a topic, do not necessarily have to be exhaustive. References to unpublished work, to documents in the grey literature (technical reports), or to any source that the reader will have difficulty finding or understanding should be avoided [27] . Most journals have reference limits and specific formatting requirements, so it is important to check the journal’s author guidelines [10] , [11] , [12] , [13] , [19] .

Authorship and acknowledgements

Determining contributors who qualify as authors and those who should be acknowledged can be difficult. Clinical Pharmacist follows guidance from the International Committee of Medical Journal Editors, which recommends that authorship be based on the following four criteria:

• Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND
• Drafting the work or revising it critically for important intellectual content; AND
• Final approval of the version to be published; AND
• Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved [3] .

Therefore, only individuals who meet all four criteria should be identified as authors [3] . The contribution of individuals who do not meet all four criteria should instead be included in the acknowledgements.

In addition, a statement that recognises any funding sources for the work should be added to the acknowledgements. This statement should adhere to the guidelines provided by the funding institution [11] .

Supplementary and supporting information

A key principle of research publication is that others should be able to replicate and build upon an author’s published claims. Therefore, submitted manuscripts should contain the necessary detail about the study and analytical design, and the data must be available for editors and peer-reviewers to allow full evaluation to take place. This is now commonplace and is seen as best practice. Author guidelines now include sections related to misconduct and falsification of data [28] . By participating in self-archiving practices and providing full data sets, authors can play their part in transparency.

The Royal Pharmaceutical Society website hosts a database to help share data from research studies. The map of evidence collates existing evidence and ongoing initiatives that can ultimately inform policy and practice relating to pharmacy; enables the sharing and showcasing of good pharmacy practice and innovation; and aims to increase the knowledge exchange and learning in pharmacy and pharmaceutical sciences [29] .

Revising your article prior to submission

Once a draft research article has been prepared, it should be shared among all of the co-authors for review and comments. A full revision of the draft should then take place to correct grammar and check flow and logic before journal submission. All authors will have to agree on the authenticity of the data and presentation before formal submission can take place [3] (see ‘Box 2: Common mistakes and reasons why research articles are rejected for publication’).

Box 2: Common mistakes and reasons why research articles are rejected for publication

• Lack of novelty and importance of the research question;
• Poor study design;
• Methods not accurately and adequately described;
• Results poorly reported, along with little analysis of data;
• Lack of statistical analysis;
• Not acknowledging the study’s limitations;
• Providing unsupported conclusions or overstating the results of the study;
• Poor writing;
• Not following the journal’s style and formatting guidance;
• Submitting a manuscript that is incomplete or outside of the aims and scope.

Selecting a journal and submitting your manuscript

It is important to select a journal for submission wisely because this choice can determine the impact and dissemination of your work [13] . Impact factor (a measure of the frequency with which the average article in a journal has been cited in a particular year), the scope and readership of a title may also influence your choice.

Furthermore, approval and adequate disclosures must be obtained from all co-authors. A conflict of interests form is also completed as part of the submissions process (normally completed by the lead author on behalf of all authors).

Many journals now request that a cover letter is also submitted to the editor, putting the study in context and explaining why the research is of importance to their audience and why it should be considered for publication in their journal.

Once this is all completed, the article can be formally submitted (usually via email or an online submission system). Figure 2 provides a sample process for a manuscript once submitted to a journal for consideration for publication.

Figure 2: Sample process for a submitted manuscript

Source: The Pharmaceutical Journal

All journals follow a similar process for article submissions, whether they use a formal online submissions system or simply email.  Clinical Pharmacist uses a process similar to this and it is useful for authors to be aware of how their submission may progress once submitted to a journal for publication.

Useful Links

• The EQUATOR Network
• Research4Life – Authorship skills modules
• Pharmacy Research UK

Reading this article counts towards your CPD

You can use the following forms to record your learning and action points from this article from Pharmaceutical Journal Publications.

Your CPD module results are stored against your account here at The Pharmaceutical Journal . You must be registered and logged into the site to do this. To review your module results, go to the ‘My Account’ tab and then ‘My CPD’.

Any training, learning or development activities that you undertake for CPD can also be recorded as evidence as part of your RPS Faculty practice-based portfolio when preparing for Faculty membership. To start your RPS Faculty journey today, access the portfolio and tools at www.rpharms.com/Faculty

If your learning was planned in advance, please click:

If your learning was spontaneous, please click:

[1] Boyer E. Scholarship reconsidered: Priorities for the professoriate . 1990. Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching.

[2] Hoogenboom BJ & Manske RC. How to write a scientific article. Int J Sports Phys Ther . 2012;7(5):512–517. PMCID: PMC3474301

[3] International Committee of Medical Journal Editors. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. 2014. Available at: http://www.icmje.org/icmje-recommendations.pdf (accessed November 2016).

[4] Dowdall M. How to be an effective peer reviewer. Clinical Pharmacist 2015;7(10). doi: 10.1211/CP.2015.20200006

[5] Nahata MC. Tips for writing and publishing an article. Ann Pharmaco . 2008;42:273–277. doi: 10.1345/aph.1K616

[6] Dixon N. Writing for publication: A guide for new authors. Int J Qual Health Care . 2001;13:417–421. doi: 10.1093/intqhc/13.5.417

[7] Keys CW. Revitalizing instruction in scientific genres: Connecting knowledge production with writing to learn in science. Sci Educ . 1999;83:115–130.

[8] Gopen G & Swan J. The science of scientific writing. Am Sci . 1990;78:550–558. Available at: http://www.americanscientist.org/issues/pub/the-science-of-scientific-writing (accessed November 2016)

[9] Hulley S, Cummings S, Browner W et al . Designing clinical research. 3rd ed. Philadelphia (PA): Lippincott Williams and Wilkins; 2007.

[10] The Pharmaceutical Journal and Clinical  Pharmacist. Author Guidance (2015). Available at: http://www.pharmaceutical-journal.com/for-authors-and-referees/article-types/#Practice_reports (accessed November 2016)

[11] Fisher JP, Jansen JA, Johnson PC et al . Guidelines for writing a research article for publication. Mary Ann Liebert Inc. Available at: https://www.liebertpub.com/media/pdf/English-Research-Article-Writing-Guide.pdf (accessed November 2016)

[12] Nature. Author Resources: How to write a paper. Available at: http://www.nature.com/authors/author_resources/how_write.html (accessed November 2016)

[13] Wiley Online Library. Resources for authors and reviewers: preparing your article. Available at: http://olabout.wiley.com/WileyCDA/Section/id-828006.html (accessed November 2016)

[14] SAGE Publications. Help readers find your article. Available at: http://www.uk.sagepub.com/journalgateway/findArticle.htm (accessed November 2016)

[15] Bem DJ. Writing the empirical journal article. In: MP Zanna & JM Darley (Eds.), The complete academic: a practical guide for the beginning social scientist (pp. 171–201). New York: Random House; 1987.

[16] Fathalla M & Fathalla M. A practical guide for health researchers. Available at: http://www.emro.who.int/dsaf/dsa237.pdf (accessed November 2016)

[17] Coghill A & Garson L (Eds.). Scientific Papers. In: A Coghill & L Garson (Eds.), The ACS Style Guide, 3 rd Edition (pp. 20–21). New York: Oxford University Press, 2006.

[18] The Scholarly Publishing and Academic Resources Institute. Available at: http://sparcopen.org/open-access/ (accessed November 2016).

[19] Elsevier. Understanding the publishing process: how to publish in scholarly journals. Available at: https://www.elsevier.com/__data/assets/pdf_file/0003/91173/Brochure_UPP_April2015.pdf (accessed November 2016).

[20] SciDevNet. How do I write a scientific paper? 2008. Available at: http://www.scidev.net/global/publishing/practical-guide/how-do-i-write-a-scientific-paper-.html (accessed November 2016)

[21] Moher D, Schultz KR & Altman DG. CONSORT GROUP (Consolidatied Standards of Reporting Trials). The CONSORT statement: Revised recommendations for improving the quality of reports of parallel‐group randomized controlled trials. Ann Intern Med . 2001;134:657–662. PMID: 11304106

[22] Bossuyt PM, Reitsma JB, Bruns DE et al . Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative. Ann Int Med 2003;138:40–44. PMID: 12513043

[23] Moher D, Liberati A, Tetzlaff J et al . The PRISMA Group (2009). Preferred reporting items for systematic reviews and meta‐analyses: the PRISMA statement. PLoS Med 6(6):e1000097. doi: 10.1371/journal.pmed1000097

[24] Devers KJ & Frankel RM. Getting qualitative research published. Educ Health 2001;14:109–117. doi: 10.1080/13576280010021888

[25] Van Way CW. Writing a scientific paper. Nutr Clin Pract 2007;22:636–640. PMID: 18042951

[26] Kishore V. When to use a survey in pharmacy practice research. The Pharmaceutical Journal 296(7886). doi: 10.1211/PJ.2016.20200700

[27] Perneger PV & Hudelson PM. Writing a research article: advice to beginners . Int J Qual Health Care 2004;16(3):191–192. doi: 10.1093/intqhc/mzh053

[28] World Association of Medical Editors. Professionalism Code of Conduct. 2016. Available at: http://www.wame.org/News/Details/16 (accessed November 2016)

[29] Royal Pharmaceutical Society. Map of Evidence. Available at: http://www.rpharms.com/support/map-of-evidence.asp (accessed November 2016)

You might also be interested in…

Working to improve our digital archive

The Pharmaceutical Journal is moving forward into a digital future

The launch of our new digital platform is just the beginning of our plans for the future of PJ

• Discoveries
• Right Journal
• Journal Metrics
• Journal Fit
• Abbreviation
• In-Text Citations
• Bibliographies
• Writing an Article
• Peer Review Types
• Acknowledgements
• Withdrawing a Paper
• Form Letter
• ISO, ANSI, CFR
• Google Scholar
• Journal Manuscript Editing
• Research Manuscript Editing

Book Editing

• Manuscript Editing Services

Medical Editing

• Bioscience Editing
• Physical Science Editing
• PhD Thesis Editing Services
• PhD Editing
• Master’s Proofreading
• Bachelor’s Editing
• Dissertation Proofreading Services
• Best Dissertation Proofreaders
• Masters Dissertation Proofreading
• PhD Proofreaders
• Proofreading PhD Thesis Price
• Journal Article Editing
• Book Editing Service
• Editing and Proofreading Services
• Research Paper Editing
• Medical Manuscript Editing
• Academic Editing
• Social Sciences Editing
• Academic Proofreading
• PhD Theses Editing
• Dissertation Proofreading
• Proofreading Rates UK
• Medical Proofreading
• PhD Proofreading Services UK
• Academic Proofreading Services UK

Medical Editing Services

• Life Science Editing
• Biomedical Editing
• Environmental Science Editing
• Pharmaceutical Science Editing
• Economics Editing
• Psychology Editing
• Sociology Editing
• Archaeology Editing
• History Paper Editing
• Anthropology Editing
• Law Paper Editing
• Engineering Paper Editing
• Technical Paper Editing
• Philosophy Editing
• PhD Dissertation Proofreading
• Lektorat Englisch
• Akademisches Lektorat
• Lektorat Englisch Preise
• Wissenschaftliches Lektorat
• Lektorat Doktorarbeit

PhD Thesis Editing

• Thesis Proofreading Services
• PhD Thesis Proofreading
• Proofreading Thesis Cost
• Proofreading Thesis
• Thesis Editing Services
• Professional Thesis Editing
• Thesis Editing Cost
• Proofreading Dissertation
• Dissertation Proofreading Cost
• Dissertation Proofreader
• Correção de Artigos Científicos
• Correção de Trabalhos Academicos
• Serviços de Correção de Inglês
• Correção de Dissertação
• Correção de Textos Precos
• 定額 ネイティブチェック
• Copy Editing
• FREE Courses
• Revision en Ingles
• Revision de Textos en Ingles
• Revision de Tesis
• Revision Medica en Ingles
• Revision de Tesis Precio
• Revisão de Artigos Científicos
• Revisão de Trabalhos Academicos
• Serviços de Revisão de Inglês
• Revisão de Dissertação
• Revisão de Textos Precos
• Corrección de Textos en Ingles
• Corrección de Tesis
• Corrección de Tesis Precio
• Corrección Medica en Ingles
• Corrector ingles

Select Page

How To Write a Journal Article

Posted by Rene Tetzner | Sep 6, 2021 | Paper Writing Advice | 0 |

How To Write a Journal Article for Publication in Twelve Essential Steps Original articles intended for publication are the most common means of disseminating the processes, results and implications of advanced research, so it is imperative that academics and scientists who wish to publish and share their work know how to write a journal article successfully. Although there are significant variations in manuscript requirements among disciplines and publishers, the writing tips I present below apply to most scholarly articles and journals across a wide range of research fields and specialisations.

Step 1: The first question to ask yourself as you begin drafting your paper or searching for a journal to publish it is what type of article will be appropriate for the material you wish to communicate. Original research, for instance, is usually reported in an original research article, whereas an evaluation of published scholarship on a topic would be written as a review article. Choosing the right type of article before you start is essential.

Step 2: Either before or after you draft your article, you will need to learn about periodicals in the field and choose one as your target journal. The scope, aims and concerns published on the journal’s website should be appropriate for your research, and the journal must publish the kind of paper necessary to communicate all important aspects of your work.

Step 3: Once you have decided on the journal to which you will be submitting your article, you should study the journal’s guidelines for authors. In some cases these will provide a great deal of information about how to write a journal article for publication; in others very little help will be offered. Either way, the guidelines must be followed with care as you prepare your article, so pay close attention to details, examples and restrictions.

Step 4: With the journal’s guidelines and your research notes by your side (or at least firmly in mind), you are ready to outline the structure and content of your article. A scientific research article is likely to use a predictable structure of introduction, literature review, methods, results, discussion and conclusions, but other fields and types of paper might offer greater freedom. Structure should never be neglected, however, since clear and logical organisation increases accessibility and reader comprehension.

Step 5: Preparing tables, figures, appendices and other supplementary materials before you actually start drafting the paper is an excellent strategy when you are struggling with how to write a journal article. The production of these tools for readers can help an author analyse and interpret findings more effectively, and writing the main text with these tools in hand tends to reduce unnecessary repetition of information.

Step 6: For many academics and scientists, starting to draft the text is the most difficult part of writing an article for publication. Beginning can be rendered easier by writing the separate sections not in the order in which they will ultimately appear, but in an order that better reflects the research process. The methods can therefore be described first, with the report of results, the discussion and the conclusions following. Once you know the contents of these parts, the introduction, background, abstract and list of references can be added.

Step 7: Be sure to take the time to assess your methods, analyse your results and interpret your findings thoroughly. Reporting what you did and what you discovered is not enough for a research paper intended for publication in a peer-reviewed journal. You will also need to tell your audience what your research means, why it is innovative and how it might be productively used by future researchers and practitioners.

Step 8: Remember as you work out how to write a journal article that there is simply no substitute for excellent writing. Scholarly prose must be both clear and correct to communicate research processes and results effectively, and the style must be both formal and appropriate for your discipline or area of specialisation. Reading published articles in your field and especially in your target journal will give you a good idea of the kind of writing you will need to submit.

Step 9: Proofread, edit and revise your draft repeatedly until you have eliminated all errors of fact, language and typing. Too many authors neglect this time-consuming aspect of how to write a journal article and suffer the consequences in the form of rejections and revision requests. As the person who knows your research better than anyone else, you are the person to ensure that your article intended for publication does that research justice.

Step 10: Recruit mentors, colleagues and friends to read your article and offer feedback. Researchers who work in your field and have successfully published their own academic or scientific writing will be able to comment constructively on research content and presentation. If English grammar, tricky references or other challenges of language and formatting prove problematic, a professional proofreader or editor can help.

Step 11: After reader feedback has been considered and the final revisions are complete, submit your manuscript exactly as the journal’s instructions indicate. Submission via an online form is an efficient and common method, but even if the journal’s preferences seem outdated, they must be observed. Unless the guidelines suggest that a cover letter to the journal editor would be unwelcome, be sure to include one to introduce your research and article in an engaging way.

Step 12: Finally, it is likely that your thoughts will be with your manuscript after you have submitted it and even that a host of ideas for further refinements will pop into your mind the instant the article is beyond your grasp. Take advantage of this impulse as you await a response by jotting your ideas down. If a request for revisions arrives from the editor, your notes are likely to prove incredibly helpful.

You might be interested in Services offered by Proof-Reading-Service.com

Journal editing.

Journal article editing services

PhD thesis editing services

Scientific Editing

Manuscript editing.

Manuscript editing services

Expert Editing

Expert editing for all papers

Research Editing

Research paper editing services

Professional book editing services

How To Write a Journal Article Reliable Tips are Designed To Help Authors Writing a Journal Article

Related Posts

How To Write the Findings Section of a Research Paper

September 2, 2021

Tips on How To Write a Journal Article

August 30, 2021

How To Write Highlights for an Academic or Scientific Paper

September 7, 2021

Tips on How To Write an Effective Figure Legend

August 27, 2021

Our Recent Posts

Our review ratings

• Examples of Research Paper Topics in Different Study Areas Score: 98%
• Dealing with Language Problems – Journal Editor’s Feedback Score: 95%
• Making Good Use of a Professional Proofreader Score: 92%
• How To Format Your Journal Paper Using Published Articles Score: 95%
• Journal Rejection as Inspiration for a New Perspective Score: 95%

Explore our Categories

• Abbreviation in Academic Writing (4)
• Career Advice for Academics (5)
• Dealing with Paper Rejection (11)
• Grammar in Academic Writing (5)
• Help with Peer Review (7)
• How To Get Published (146)
• Paper Writing Advice (17)
• Referencing & Bibliographies (16)

How to find the right journal for your research (using actual data)

Joanna Wilkinson

Want to help your research flourish? We share tips for using publisher-neutral data and statistics to find the right journal for your research paper.

The right journal helps your research flourish. It puts you in the best position to reach a relevant and engaged audience, and can extend the impact of your paper through a high-quality publishing process.

Unfortunately, finding the right journal is a particular pain point for inexperienced authors and those who publish on interdisciplinary topics. The sheer number of journals published today is one reason for this. More than 42,000 active scholarly peer-reviewed journals were published in 2018 alone, and there’s been accelerated growth of more than 5% in recent years.

The overwhelming growth in journals has left many researchers struggling to find the best home for their manuscripts which can be a daunting prospect after several long months producing research. Submitting to the wrong journal can hinder the impact of your manuscript. It could even result in a series of rejections, stalling both your research and career. Conversely, the right journal can help you showcase your research to the world in an environment consistent with your values.

Keep reading to learn how solutions like Journal Citation Reports ™ (JCR) and Master Journal List   can help you find the right journal for your research in the fastest possible time.

What to look for in a journal and why

To find the right journal for your research paper, it’s important to consider what you need and want out of the publishing process.

The goal for many researchers is to find a prestigious, peer-reviewed journal to publish in. This might be one that can support an application for tenure, promotion or future funding. It’s not always that simple, however. If your research is in a specialized field, you may want to avoid a journal with a multidisciplinary focus. And if you have ground-breaking results, you may want to pay attention to journals with a speedy review process and frequent publication schedule. Moreover, you may want to publish your paper as open access so that it’s accessible to everyone—and your institution or funder may also require this.

With so many points to consider, it’s good practice to have a journal in mind before you start writing. We published an earlier post to help you with this: Find top journals in a research field, step-by-step guide . Check it out to discover where the top researchers in your field are publishing.

Already written your manuscript? No problem: this blog will help you use publisher-neutral data and statistics to choose the right journal for your paper.

First stop: Manuscript Matcher in the Master Journal List

Master Journal List Manuscript Matcher is the ultimate place to begin your search for journals. It is a free tool that helps you narrow down your journal options based on your research topic and goals.

Pairing your research with a journal

Manuscript Matcher, also available via EndNote™ , provides a list of relevant journals indexed in the Web of Science™ . First, you’ll want to input your title and abstract (or keywords, if you prefer). You can then filter your results using the options shown on the left-hand sidebar, or simply click on the profile page of any journal listed.

Each journal page details the journal’s coverage in the Web of Science. Where available, it may also display a wealth of information, including:

• open access information (including whether a journal is Gold OA)
• the journal’s aims and scope
• download statistics
• average number of weeks from submission to publication, and
• peer review information (including type and policy)

Ready to try Manuscript Matcher? Follow this link .

Identify the journals that are a good topical fit for your research using Manuscript Matcher. You can then move to Journal Citation Reports to understand their citation impact, audience and open access statistics.

Find the right journal with Journal Citation Reports

Journal Citation Reports   is the most powerful solution for journal intelligence. It uses transparent, publisher-neutral data and statistics to provide unique insight into a journal’s role and influence. This will help you produce a definitive list of journals best-placed to publish your findings, and more.

Three data points exist on every journal page to help you assess a journal as a home for your research. These are: citation metrics, article relevance and audience.

Citation Metrics

The Journal Impact Factor™ (JIF) is included as part of the rich array of citation metrics offered on each journal page. It shows how often a journal’s recently published material is cited on average.

Learn how the JIF is calculated in this guide .

It’s important to note that the JIF has its limitations and no researcher should depend on the impact factor alone when assessing the usefulness or prestige of a journal. Journal Citation Reports helps you understand the context of a journal’s JIF and how to use the metric responsibly.

The JCR Trend Graph, for example, places the JIF in the context of time and subject category performance. Citation behavior varies across disciplines, and journals in JCR may be placed across multiple subject categories depending on the scope of their content. The Trend Graph shows you how the journal performs against others in the same subject category. It also gives you an understanding of how stable that performance is year-on-year.

You can learn more about this here .

The 2021 JCR release introduced a new, field-normalized metric for measuring the citation impact of a journal’s recent publications. By normalizing for different fields of research and their widely varying rates of publication and citation, the Journal Citation Indicator provides a single journal-level metric that can be easily interpreted and compared across disciplines. Learn more about the Journal Citation Indicator here .

Article relevance

The Contributing Items section in JCR demonstrates whether the journal is a good match for your paper. It can also validate the information you found in the Manuscript Matcher. You can view the full list in the Web of Science by selecting “Show all.”

JCR helps you understand the scholarly community engaging with a journal on both a country and an institutional level. This information provides insight on where in the world your own paper might have an impact if published in that particular journal. It also gives you a sense of general readership, and who you might be talking to if you choose that journal.

Start using Journal Citation Reports today .

Ready to find the right journal for your paper?

The expansion of scholarly journals in previous years has made it difficult for researchers to choose the right journal for their research. This isn’t a good position to be in when you’ve spent many long months preparing your research for the world. Journal Citation Reports , Manuscript Matcher by Master Journal List  and the Web of Science  are all products dedicated to helping you find the right home for your paper. Try them out today and help your research flourish.

Stay connected

Want to learn more?  You can also read related articles in our Research Smarter series,  with guidance on finding the relevant papers for your research  and how you can save hundreds of hours in the writing process . You can also read about the 2022 JCR release here . Finally, subscribe to receive our latest news, resources and events to help make your research journey a smart one.

Subscribe to receive regular updates on how to research smarter

Related posts

Demonstrating socioeconomic impact – a historical perspective of ancient wisdom and modern challenges.

Unlocking U.K. research excellence: Key insights from the Research Professional News Live summit

For better insights, assess research performance at the department level

Explore millions of high-quality primary sources and images from around the world, including artworks, maps, photographs, and more.

Explore migration issues through a variety of media types

• Part of The Streets are Talking: Public Forms of Creative Expression from Around the World
• Part of The Journal of Economic Perspectives, Vol. 34, No. 1 (Winter 2020)
• Part of Cato Institute (Aug. 3, 2021)
• Part of University of California Press
• Part of Open: Smithsonian National Museum of African American History & Culture
• Part of Indiana Journal of Global Legal Studies, Vol. 19, No. 1 (Winter 2012)
• Part of R Street Institute (Nov. 1, 2020)
• Part of Leuven University Press
• Part of UN Secretary-General Papers: Ban Ki-moon (2007-2016)
• Part of Perspectives on Terrorism, Vol. 12, No. 4 (August 2018)
• Part of Leveraging Lives: Serbia and Illegal Tunisian Migration to Europe, Carnegie Endowment for International Peace (Mar. 1, 2023)
• Part of UCL Press

Harness the power of visual materials—explore more than 3 million images now on JSTOR.

Enhance your scholarly research with underground newspapers, magazines, and journals.

Explore collections in the arts, sciences, and literature from the world’s leading museums, archives, and scholars.

• Search Menu
• Advance Articles
• Supplements
• Virtual Collections
• Author Guidelines
• Submission Site
• Open Access
• Self-Archiving Policy
• Benefits of Publishing with JBCR
• About Journal of Burn Care and Research
• About the American Burn Association
• Editorial Board
• Advertising & Corporate Services
• Journals Career Network
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

• < Previous

602 Too Few Burn Papers and Images, Lowering Publication Barriers for Low- and Middle-Income Country Authors

• Article contents
• Figures & tables
• Supplementary Data

Cameron J Kneib, Ryann E Shor, Kiran K Nakarmi, Manish K Yadav, Raslina Shrestha, Ariel Miranda, Tam N Pham, Barclay T Stewart, 602 Too Few Burn Papers and Images, Lowering Publication Barriers for Low- and Middle-Income Country Authors, Journal of Burn Care & Research , Volume 45, Issue Supplement_1, May/June 2024, Page 185, https://doi.org/10.1093/jbcr/irae036.236

• Permissions Icon Permissions

Whereas the burden of burn injury is greatest in low- and middle-income countries (LMIC), publications from these regions represent a disproportionately small volume of the published burn literature in English. This important gap in our cumulative knowledge contributes to the slow progress in global burn injury control. In addition to manuscripts, clinical photographs are important representations of injury and outcomes. We aimed to compare the current volume of burn publications and photographs to the global burn injury burden and sought to identify potential journal-level barriers to publication.

A bibliometric analysis was performed by sampling three leading burn surgery journals published in the English language (Journal of Burn Care and Research, Burns, European Burn Journal) between 2021-2023. Country of corresponding author and number of photos were tabulated and categorized by region. Volume was compared to Global Burden of Disease (GBD) 2019 estimates for incidence of thermal injury by region. Journal submission instructions were reviewed for potential barriers to publication.

A total of 916 articles were reviewed from 59 countries. Analysis showed 147 articles with 1117 human photos. GBD incidence of burn, publications, and photographs by region are shown in Figure 1. North America, Europe, the Middle East, and Oceania had a higher proportion of published images to burn injury incidence than Latin America, Africa, and Asia (X2 (6, N = 915) =3585, p < 0.001). Barriers to journal publication included charges for published images (up to 600 USD/image), open access costs (900-3500 USD with 1 of 3 publishers offering tiered discounts specific to LMIC), English language review (0.06-0.07 USD/word), and society affiliation discounts. Table 1 summarizes identified barriers and potential solutions.

There is a relative paucity of LMIC contributions to publications and clinical photos in English language burn journals. Identified publication barriers and added costs from journals likely contribute to this disparity.

Publication-level strategies to increase authorship and clinical photographs from underrepresented regions are urgently needed.

Email alerts

Citing articles via, looking for your next opportunity.

• Recommend to Your Librarian
• Advertising and Corporate Services

Affiliations

• Online ISSN 1559-0488
• Print ISSN 1559-047X
• Copyright © 2024 American Burn Association
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

ORIGINAL RESEARCH article

An inflammation-related subtype classification for analyzing tumor microenvironment and clinical prognosis in colorectal cancer.

• Beijing Cancer Hospital, Peking University, Beijing, China

The final, formatted version of the article will be published soon.

Select one of your emails

You have multiple emails registered with Frontiers:

Notify me on publication

Please enter your email address:

If you already have an account, please login

You don't have a Frontiers account ? You can register here

Background The inflammatory response plays an essential role in the tumor microenvironment (TME) of colorectal cancer (CRC) by modulating tumor growth, progression, and response to therapy through the recruitment of immune cells, production of cytokines, and activation of signaling pathways. However, the molecular subtypes and risk score prognostic model based on inflammatory response remain to be further explored. Methods Inflammation-related genes were collected from the molecular signature database and molecular subtypes were identified using nonnegative matrix factorization based on the TCGA cohort. We compared the clinicopathological features, immune infiltration, somatic mutation profile, survival prognosis, and drug sensitivity between the subtypes. The risk score model was developed using LASSO and multivariate Cox regression in the TCGA cohort. The above results were independently validated in the GEO cohort. Moreover, we explored the biological functions of the hub gene, receptor interacting protein kinase 2 (RIPK2), leveraging proteomics data, in vivo, and in vitro experiments. Results We identified two inflammation-related subtypes (inflammation-low and inflammation-high) and have excellent internal consistency and stability. Inflammation-high subtype showed higher immune cell infiltration and increased sensitivity to common chemotherapeutic drugs, while inflammation-low subtype may be more suitable for immunotherapy. Besides, the two subtypes differ significantly in pathway enrichment and biological functions. In addition, the 11-gene signature prognostic model constructed from inflammation-related genes showed strong prognostic assessment power and could serve as a novel prognostic marker to predict the survival of CRC patients. Finally, RIPK2 plays a crucial role in promoting malignant proliferation of CRC cell validated by experiment. Conclusions This study provides new insights into the heterogeneity of CRC and provides novel opportunities for treatment development and clinical decision making. Keywords Inflammatory response, colorectal cancer, molecular subtype, prognosis, tumor microenvironment.

Keywords: Inflammatory Response, colorectal cancer, Molecular subtype, prognosis, Tumor Microenvironment

Received: 31 Jan 2024; Accepted: 15 Apr 2024.

Copyright: © 2024 Pei, Gao and Wu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Aiwen Wu, Beijing Cancer Hospital, Peking University, Beijing, China

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.25(3); 2014 Oct

Peer Review in Scientific Publications: Benefits, Critiques, & A Survival Guide

Jacalyn kelly.

1 Clinical Biochemistry, Department of Pediatric Laboratory Medicine, The Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada

Tara Sadeghieh

Khosrow adeli.

2 Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada

3 Chair, Communications and Publications Division (CPD), International Federation for Sick Clinical Chemistry (IFCC), Milan, Italy

The authors declare no conflicts of interest regarding publication of this article.

Peer review has been defined as a process of subjecting an author’s scholarly work, research or ideas to the scrutiny of others who are experts in the same field. It functions to encourage authors to meet the accepted high standards of their discipline and to control the dissemination of research data to ensure that unwarranted claims, unacceptable interpretations or personal views are not published without prior expert review. Despite its wide-spread use by most journals, the peer review process has also been widely criticised due to the slowness of the process to publish new findings and due to perceived bias by the editors and/or reviewers. Within the scientific community, peer review has become an essential component of the academic writing process. It helps ensure that papers published in scientific journals answer meaningful research questions and draw accurate conclusions based on professionally executed experimentation. Submission of low quality manuscripts has become increasingly prevalent, and peer review acts as a filter to prevent this work from reaching the scientific community. The major advantage of a peer review process is that peer-reviewed articles provide a trusted form of scientific communication. Since scientific knowledge is cumulative and builds on itself, this trust is particularly important. Despite the positive impacts of peer review, critics argue that the peer review process stifles innovation in experimentation, and acts as a poor screen against plagiarism. Despite its downfalls, there has not yet been a foolproof system developed to take the place of peer review, however, researchers have been looking into electronic means of improving the peer review process. Unfortunately, the recent explosion in online only/electronic journals has led to mass publication of a large number of scientific articles with little or no peer review. This poses significant risk to advances in scientific knowledge and its future potential. The current article summarizes the peer review process, highlights the pros and cons associated with different types of peer review, and describes new methods for improving peer review.

WHAT IS PEER REVIEW AND WHAT IS ITS PURPOSE?

Peer Review is defined as “a process of subjecting an author’s scholarly work, research or ideas to the scrutiny of others who are experts in the same field” ( 1 ). Peer review is intended to serve two primary purposes. Firstly, it acts as a filter to ensure that only high quality research is published, especially in reputable journals, by determining the validity, significance and originality of the study. Secondly, peer review is intended to improve the quality of manuscripts that are deemed suitable for publication. Peer reviewers provide suggestions to authors on how to improve the quality of their manuscripts, and also identify any errors that need correcting before publication.

HISTORY OF PEER REVIEW

The concept of peer review was developed long before the scholarly journal. In fact, the peer review process is thought to have been used as a method of evaluating written work since ancient Greece ( 2 ). The peer review process was first described by a physician named Ishaq bin Ali al-Rahwi of Syria, who lived from 854-931 CE, in his book Ethics of the Physician ( 2 ). There, he stated that physicians must take notes describing the state of their patients’ medical conditions upon each visit. Following treatment, the notes were scrutinized by a local medical council to determine whether the physician had met the required standards of medical care. If the medical council deemed that the appropriate standards were not met, the physician in question could receive a lawsuit from the maltreated patient ( 2 ).

The invention of the printing press in 1453 allowed written documents to be distributed to the general public ( 3 ). At this time, it became more important to regulate the quality of the written material that became publicly available, and editing by peers increased in prevalence. In 1620, Francis Bacon wrote the work Novum Organum, where he described what eventually became known as the first universal method for generating and assessing new science ( 3 ). His work was instrumental in shaping the Scientific Method ( 3 ). In 1665, the French Journal des sçavans and the English Philosophical Transactions of the Royal Society were the first scientific journals to systematically publish research results ( 4 ). Philosophical Transactions of the Royal Society is thought to be the first journal to formalize the peer review process in 1665 ( 5 ), however, it is important to note that peer review was initially introduced to help editors decide which manuscripts to publish in their journals, and at that time it did not serve to ensure the validity of the research ( 6 ). It did not take long for the peer review process to evolve, and shortly thereafter papers were distributed to reviewers with the intent of authenticating the integrity of the research study before publication. The Royal Society of Edinburgh adhered to the following peer review process, published in their Medical Essays and Observations in 1731: “Memoirs sent by correspondence are distributed according to the subject matter to those members who are most versed in these matters. The report of their identity is not known to the author.” ( 7 ). The Royal Society of London adopted this review procedure in 1752 and developed the “Committee on Papers” to review manuscripts before they were published in Philosophical Transactions ( 6 ).

Peer review in the systematized and institutionalized form has developed immensely since the Second World War, at least partly due to the large increase in scientific research during this period ( 7 ). It is now used not only to ensure that a scientific manuscript is experimentally and ethically sound, but also to determine which papers sufficiently meet the journal’s standards of quality and originality before publication. Peer review is now standard practice by most credible scientific journals, and is an essential part of determining the credibility and quality of work submitted.

IMPACT OF THE PEER REVIEW PROCESS

Peer review has become the foundation of the scholarly publication system because it effectively subjects an author’s work to the scrutiny of other experts in the field. Thus, it encourages authors to strive to produce high quality research that will advance the field. Peer review also supports and maintains integrity and authenticity in the advancement of science. A scientific hypothesis or statement is generally not accepted by the academic community unless it has been published in a peer-reviewed journal ( 8 ). The Institute for Scientific Information ( ISI ) only considers journals that are peer-reviewed as candidates to receive Impact Factors. Peer review is a well-established process which has been a formal part of scientific communication for over 300 years.

OVERVIEW OF THE PEER REVIEW PROCESS

The peer review process begins when a scientist completes a research study and writes a manuscript that describes the purpose, experimental design, results, and conclusions of the study. The scientist then submits this paper to a suitable journal that specializes in a relevant research field, a step referred to as pre-submission. The editors of the journal will review the paper to ensure that the subject matter is in line with that of the journal, and that it fits with the editorial platform. Very few papers pass this initial evaluation. If the journal editors feel the paper sufficiently meets these requirements and is written by a credible source, they will send the paper to accomplished researchers in the field for a formal peer review. Peer reviewers are also known as referees (this process is summarized in Figure 1 ). The role of the editor is to select the most appropriate manuscripts for the journal, and to implement and monitor the peer review process. Editors must ensure that peer reviews are conducted fairly, and in an effective and timely manner. They must also ensure that there are no conflicts of interest involved in the peer review process.

Overview of the review process

When a reviewer is provided with a paper, he or she reads it carefully and scrutinizes it to evaluate the validity of the science, the quality of the experimental design, and the appropriateness of the methods used. The reviewer also assesses the significance of the research, and judges whether the work will contribute to advancement in the field by evaluating the importance of the findings, and determining the originality of the research. Additionally, reviewers identify any scientific errors and references that are missing or incorrect. Peer reviewers give recommendations to the editor regarding whether the paper should be accepted, rejected, or improved before publication in the journal. The editor will mediate author-referee discussion in order to clarify the priority of certain referee requests, suggest areas that can be strengthened, and overrule reviewer recommendations that are beyond the study’s scope ( 9 ). If the paper is accepted, as per suggestion by the peer reviewer, the paper goes into the production stage, where it is tweaked and formatted by the editors, and finally published in the scientific journal. An overview of the review process is presented in Figure 1 .

WHO CONDUCTS REVIEWS?

Peer reviews are conducted by scientific experts with specialized knowledge on the content of the manuscript, as well as by scientists with a more general knowledge base. Peer reviewers can be anyone who has competence and expertise in the subject areas that the journal covers. Reviewers can range from young and up-and-coming researchers to old masters in the field. Often, the young reviewers are the most responsive and deliver the best quality reviews, though this is not always the case. On average, a reviewer will conduct approximately eight reviews per year, according to a study on peer review by the Publishing Research Consortium (PRC) ( 7 ). Journals will often have a pool of reviewers with diverse backgrounds to allow for many different perspectives. They will also keep a rather large reviewer bank, so that reviewers do not get burnt out, overwhelmed or time constrained from reviewing multiple articles simultaneously.

WHY DO REVIEWERS REVIEW?

Referees are typically not paid to conduct peer reviews and the process takes considerable effort, so the question is raised as to what incentive referees have to review at all. Some feel an academic duty to perform reviews, and are of the mentality that if their peers are expected to review their papers, then they should review the work of their peers as well. Reviewers may also have personal contacts with editors, and may want to assist as much as possible. Others review to keep up-to-date with the latest developments in their field, and reading new scientific papers is an effective way to do so. Some scientists use peer review as an opportunity to advance their own research as it stimulates new ideas and allows them to read about new experimental techniques. Other reviewers are keen on building associations with prestigious journals and editors and becoming part of their community, as sometimes reviewers who show dedication to the journal are later hired as editors. Some scientists see peer review as a chance to become aware of the latest research before their peers, and thus be first to develop new insights from the material. Finally, in terms of career development, peer reviewing can be desirable as it is often noted on one’s resume or CV. Many institutions consider a researcher’s involvement in peer review when assessing their performance for promotions ( 11 ). Peer reviewing can also be an effective way for a scientist to show their superiors that they are committed to their scientific field ( 5 ).

ARE REVIEWERS KEEN TO REVIEW?

A 2009 international survey of 4000 peer reviewers conducted by the charity Sense About Science at the British Science Festival at the University of Surrey, found that 90% of reviewers were keen to peer review ( 12 ). One third of respondents to the survey said they were happy to review up to five papers per year, and an additional one third of respondents were happy to review up to ten.

HOW LONG DOES IT TAKE TO REVIEW ONE PAPER?

On average, it takes approximately six hours to review one paper ( 12 ), however, this number may vary greatly depending on the content of the paper and the nature of the peer reviewer. One in every 100 participants in the “Sense About Science” survey claims to have taken more than 100 hours to review their last paper ( 12 ).

HOW TO DETERMINE IF A JOURNAL IS PEER REVIEWED

Ulrichsweb is a directory that provides information on over 300,000 periodicals, including information regarding which journals are peer reviewed ( 13 ). After logging into the system using an institutional login (eg. from the University of Toronto), search terms, journal titles or ISSN numbers can be entered into the search bar. The database provides the title, publisher, and country of origin of the journal, and indicates whether the journal is still actively publishing. The black book symbol (labelled ‘refereed’) reveals that the journal is peer reviewed.

THE EVALUATION CRITERIA FOR PEER REVIEW OF SCIENTIFIC PAPERS

As previously mentioned, when a reviewer receives a scientific manuscript, he/she will first determine if the subject matter is well suited for the content of the journal. The reviewer will then consider whether the research question is important and original, a process which may be aided by a literature scan of review articles.

Scientific papers submitted for peer review usually follow a specific structure that begins with the title, followed by the abstract, introduction, methodology, results, discussion, conclusions, and references. The title must be descriptive and include the concept and organism investigated, and potentially the variable manipulated and the systems used in the study. The peer reviewer evaluates if the title is descriptive enough, and ensures that it is clear and concise. A study by the National Association of Realtors (NAR) published by the Oxford University Press in 2006 indicated that the title of a manuscript plays a significant role in determining reader interest, as 72% of respondents said they could usually judge whether an article will be of interest to them based on the title and the author, while 13% of respondents claimed to always be able to do so ( 14 ).

The abstract is a summary of the paper, which briefly mentions the background or purpose, methods, key results, and major conclusions of the study. The peer reviewer assesses whether the abstract is sufficiently informative and if the content of the abstract is consistent with the rest of the paper. The NAR study indicated that 40% of respondents could determine whether an article would be of interest to them based on the abstract alone 60-80% of the time, while 32% could judge an article based on the abstract 80-100% of the time ( 14 ). This demonstrates that the abstract alone is often used to assess the value of an article.

The introduction of a scientific paper presents the research question in the context of what is already known about the topic, in order to identify why the question being studied is of interest to the scientific community, and what gap in knowledge the study aims to fill ( 15 ). The introduction identifies the study’s purpose and scope, briefly describes the general methods of investigation, and outlines the hypothesis and predictions ( 15 ). The peer reviewer determines whether the introduction provides sufficient background information on the research topic, and ensures that the research question and hypothesis are clearly identifiable.

The methods section describes the experimental procedures, and explains why each experiment was conducted. The methods section also includes the equipment and reagents used in the investigation. The methods section should be detailed enough that it can be used it to repeat the experiment ( 15 ). Methods are written in the past tense and in the active voice. The peer reviewer assesses whether the appropriate methods were used to answer the research question, and if they were written with sufficient detail. If information is missing from the methods section, it is the peer reviewer’s job to identify what details need to be added.

The results section is where the outcomes of the experiment and trends in the data are explained without judgement, bias or interpretation ( 15 ). This section can include statistical tests performed on the data, as well as figures and tables in addition to the text. The peer reviewer ensures that the results are described with sufficient detail, and determines their credibility. Reviewers also confirm that the text is consistent with the information presented in tables and figures, and that all figures and tables included are important and relevant ( 15 ). The peer reviewer will also make sure that table and figure captions are appropriate both contextually and in length, and that tables and figures present the data accurately.

The discussion section is where the data is analyzed. Here, the results are interpreted and related to past studies ( 15 ). The discussion describes the meaning and significance of the results in terms of the research question and hypothesis, and states whether the hypothesis was supported or rejected. This section may also provide possible explanations for unusual results and suggestions for future research ( 15 ). The discussion should end with a conclusions section that summarizes the major findings of the investigation. The peer reviewer determines whether the discussion is clear and focused, and whether the conclusions are an appropriate interpretation of the results. Reviewers also ensure that the discussion addresses the limitations of the study, any anomalies in the results, the relationship of the study to previous research, and the theoretical implications and practical applications of the study.

The references are found at the end of the paper, and list all of the information sources cited in the text to describe the background, methods, and/or interpret results. Depending on the citation method used, the references are listed in alphabetical order according to author last name, or numbered according to the order in which they appear in the paper. The peer reviewer ensures that references are used appropriately, cited accurately, formatted correctly, and that none are missing.

Finally, the peer reviewer determines whether the paper is clearly written and if the content seems logical. After thoroughly reading through the entire manuscript, they determine whether it meets the journal’s standards for publication,

and whether it falls within the top 25% of papers in its field ( 16 ) to determine priority for publication. An overview of what a peer reviewer looks for when evaluating a manuscript, in order of importance, is presented in Figure 2 .

How a peer review evaluates a manuscript

To increase the chance of success in the peer review process, the author must ensure that the paper fully complies with the journal guidelines before submission. The author must also be open to criticism and suggested revisions, and learn from mistakes made in previous submissions.

ADVANTAGES AND DISADVANTAGES OF THE DIFFERENT TYPES OF PEER REVIEW

The peer review process is generally conducted in one of three ways: open review, single-blind review, or double-blind review. In an open review, both the author of the paper and the peer reviewer know one another’s identity. Alternatively, in single-blind review, the reviewer’s identity is kept private, but the author’s identity is revealed to the reviewer. In double-blind review, the identities of both the reviewer and author are kept anonymous. Open peer review is advantageous in that it prevents the reviewer from leaving malicious comments, being careless, or procrastinating completion of the review ( 2 ). It encourages reviewers to be open and honest without being disrespectful. Open reviewing also discourages plagiarism amongst authors ( 2 ). On the other hand, open peer review can also prevent reviewers from being honest for fear of developing bad rapport with the author. The reviewer may withhold or tone down their criticisms in order to be polite ( 2 ). This is especially true when younger reviewers are given a more esteemed author’s work, in which case the reviewer may be hesitant to provide criticism for fear that it will damper their relationship with a superior ( 2 ). According to the Sense About Science survey, editors find that completely open reviewing decreases the number of people willing to participate, and leads to reviews of little value ( 12 ). In the aforementioned study by the PRC, only 23% of authors surveyed had experience with open peer review ( 7 ).

Single-blind peer review is by far the most common. In the PRC study, 85% of authors surveyed had experience with single-blind peer review ( 7 ). This method is advantageous as the reviewer is more likely to provide honest feedback when their identity is concealed ( 2 ). This allows the reviewer to make independent decisions without the influence of the author ( 2 ). The main disadvantage of reviewer anonymity, however, is that reviewers who receive manuscripts on subjects similar to their own research may be tempted to delay completing the review in order to publish their own data first ( 2 ).

Double-blind peer review is advantageous as it prevents the reviewer from being biased against the author based on their country of origin or previous work ( 2 ). This allows the paper to be judged based on the quality of the content, rather than the reputation of the author. The Sense About Science survey indicates that 76% of researchers think double-blind peer review is a good idea ( 12 ), and the PRC survey indicates that 45% of authors have had experience with double-blind peer review ( 7 ). The disadvantage of double-blind peer review is that, especially in niche areas of research, it can sometimes be easy for the reviewer to determine the identity of the author based on writing style, subject matter or self-citation, and thus, impart bias ( 2 ).

Masking the author’s identity from peer reviewers, as is the case in double-blind review, is generally thought to minimize bias and maintain review quality. A study by Justice et al. in 1998 investigated whether masking author identity affected the quality of the review ( 17 ). One hundred and eighteen manuscripts were randomized; 26 were peer reviewed as normal, and 92 were moved into the ‘intervention’ arm, where editor quality assessments were completed for 77 manuscripts and author quality assessments were completed for 40 manuscripts ( 17 ). There was no perceived difference in quality between the masked and unmasked reviews. Additionally, the masking itself was often unsuccessful, especially with well-known authors ( 17 ). However, a previous study conducted by McNutt et al. had different results ( 18 ). In this case, blinding was successful 73% of the time, and they found that when author identity was masked, the quality of review was slightly higher ( 18 ). Although Justice et al. argued that this difference was too small to be consequential, their study targeted only biomedical journals, and the results cannot be generalized to journals of a different subject matter ( 17 ). Additionally, there were problems masking the identities of well-known authors, introducing a flaw in the methods. Regardless, Justice et al. concluded that masking author identity from reviewers may not improve review quality ( 17 ).

In addition to open, single-blind and double-blind peer review, there are two experimental forms of peer review. In some cases, following publication, papers may be subjected to post-publication peer review. As many papers are now published online, the scientific community has the opportunity to comment on these papers, engage in online discussions and post a formal review. For example, online publishers PLOS and BioMed Central have enabled scientists to post comments on published papers if they are registered users of the site ( 10 ). Philica is another journal launched with this experimental form of peer review. Only 8% of authors surveyed in the PRC study had experience with post-publication review ( 7 ). Another experimental form of peer review called Dynamic Peer Review has also emerged. Dynamic peer review is conducted on websites such as Naboj, which allow scientists to conduct peer reviews on articles in the preprint media ( 19 ). The peer review is conducted on repositories and is a continuous process, which allows the public to see both the article and the reviews as the article is being developed ( 19 ). Dynamic peer review helps prevent plagiarism as the scientific community will already be familiar with the work before the peer reviewed version appears in print ( 19 ). Dynamic review also reduces the time lag between manuscript submission and publishing. An example of a preprint server is the ‘arXiv’ developed by Paul Ginsparg in 1991, which is used primarily by physicists ( 19 ). These alternative forms of peer review are still un-established and experimental. Traditional peer review is time-tested and still highly utilized. All methods of peer review have their advantages and deficiencies, and all are prone to error.

PEER REVIEW OF OPEN ACCESS JOURNALS

Open access (OA) journals are becoming increasingly popular as they allow the potential for widespread distribution of publications in a timely manner ( 20 ). Nevertheless, there can be issues regarding the peer review process of open access journals. In a study published in Science in 2013, John Bohannon submitted 304 slightly different versions of a fictional scientific paper (written by a fake author, working out of a non-existent institution) to a selected group of OA journals. This study was performed in order to determine whether papers submitted to OA journals are properly reviewed before publication in comparison to subscription-based journals. The journals in this study were selected from the Directory of Open Access Journals (DOAJ) and Biall’s List, a list of journals which are potentially predatory, and all required a fee for publishing ( 21 ). Of the 304 journals, 157 accepted a fake paper, suggesting that acceptance was based on financial interest rather than the quality of article itself, while 98 journals promptly rejected the fakes ( 21 ). Although this study highlights useful information on the problems associated with lower quality publishers that do not have an effective peer review system in place, the article also generalizes the study results to all OA journals, which can be detrimental to the general perception of OA journals. There were two limitations of the study that made it impossible to accurately determine the relationship between peer review and OA journals: 1) there was no control group (subscription-based journals), and 2) the fake papers were sent to a non-randomized selection of journals, resulting in bias.

JOURNAL ACCEPTANCE RATES

Based on a recent survey, the average acceptance rate for papers submitted to scientific journals is about 50% ( 7 ). Twenty percent of the submitted manuscripts that are not accepted are rejected prior to review, and 30% are rejected following review ( 7 ). Of the 50% accepted, 41% are accepted with the condition of revision, while only 9% are accepted without the request for revision ( 7 ).

SATISFACTION WITH THE PEER REVIEW SYSTEM

Based on a recent survey by the PRC, 64% of academics are satisfied with the current system of peer review, and only 12% claimed to be ‘dissatisfied’ ( 7 ). The large majority, 85%, agreed with the statement that ‘scientific communication is greatly helped by peer review’ ( 7 ). There was a similarly high level of support (83%) for the idea that peer review ‘provides control in scientific communication’ ( 7 ).

HOW TO PEER REVIEW EFFECTIVELY

The following are ten tips on how to be an effective peer reviewer as indicated by Brian Lucey, an expert on the subject ( 22 ):

1) Be professional

Peer review is a mutual responsibility among fellow scientists, and scientists are expected, as part of the academic community, to take part in peer review. If one is to expect others to review their work, they should commit to reviewing the work of others as well, and put effort into it.

2) Be pleasant

If the paper is of low quality, suggest that it be rejected, but do not leave ad hominem comments. There is no benefit to being ruthless.

3) Read the invite

When emailing a scientist to ask them to conduct a peer review, the majority of journals will provide a link to either accept or reject. Do not respond to the email, respond to the link.

4) Be helpful

Suggest how the authors can overcome the shortcomings in their paper. A review should guide the author on what is good and what needs work from the reviewer’s perspective.

5) Be scientific

The peer reviewer plays the role of a scientific peer, not an editor for proofreading or decision-making. Don’t fill a review with comments on editorial and typographic issues. Instead, focus on adding value with scientific knowledge and commenting on the credibility of the research conducted and conclusions drawn. If the paper has a lot of typographical errors, suggest that it be professionally proof edited as part of the review.

6) Be timely

Stick to the timeline given when conducting a peer review. Editors track who is reviewing what and when and will know if someone is late on completing a review. It is important to be timely both out of respect for the journal and the author, as well as to not develop a reputation of being late for review deadlines.

7) Be realistic

The peer reviewer must be realistic about the work presented, the changes they suggest and their role. Peer reviewers may set the bar too high for the paper they are editing by proposing changes that are too ambitious and editors must override them.

8) Be empathetic

Ensure that the review is scientific, helpful and courteous. Be sensitive and respectful with word choice and tone in a review.

Remember that both specialists and generalists can provide valuable insight when peer reviewing. Editors will try to get both specialised and general reviewers for any particular paper to allow for different perspectives. If someone is asked to review, the editor has determined they have a valid and useful role to play, even if the paper is not in their area of expertise.

10) Be organised

A review requires structure and logical flow. A reviewer should proofread their review before submitting it for structural, grammatical and spelling errors as well as for clarity. Most publishers provide short guides on structuring a peer review on their website. Begin with an overview of the proposed improvements; then provide feedback on the paper structure, the quality of data sources and methods of investigation used, the logical flow of argument, and the validity of conclusions drawn. Then provide feedback on style, voice and lexical concerns, with suggestions on how to improve.

In addition, the American Physiology Society (APS) recommends in its Peer Review 101 Handout that peer reviewers should put themselves in both the editor’s and author’s shoes to ensure that they provide what both the editor and the author need and expect ( 11 ). To please the editor, the reviewer should ensure that the peer review is completed on time, and that it provides clear explanations to back up recommendations. To be helpful to the author, the reviewer must ensure that their feedback is constructive. It is suggested that the reviewer take time to think about the paper; they should read it once, wait at least a day, and then re-read it before writing the review ( 11 ). The APS also suggests that Graduate students and researchers pay attention to how peer reviewers edit their work, as well as to what edits they find helpful, in order to learn how to peer review effectively ( 11 ). Additionally, it is suggested that Graduate students practice reviewing by editing their peers’ papers and asking a faculty member for feedback on their efforts. It is recommended that young scientists offer to peer review as often as possible in order to become skilled at the process ( 11 ). The majority of students, fellows and trainees do not get formal training in peer review, but rather learn by observing their mentors. According to the APS, one acquires experience through networking and referrals, and should therefore try to strengthen relationships with journal editors by offering to review manuscripts ( 11 ). The APS also suggests that experienced reviewers provide constructive feedback to students and junior colleagues on their peer review efforts, and encourages them to peer review to demonstrate the importance of this process in improving science ( 11 ).

The peer reviewer should only comment on areas of the manuscript that they are knowledgeable about ( 23 ). If there is any section of the manuscript they feel they are not qualified to review, they should mention this in their comments and not provide further feedback on that section. The peer reviewer is not permitted to share any part of the manuscript with a colleague (even if they may be more knowledgeable in the subject matter) without first obtaining permission from the editor ( 23 ). If a peer reviewer comes across something they are unsure of in the paper, they can consult the literature to try and gain insight. It is important for scientists to remember that if a paper can be improved by the expertise of one of their colleagues, the journal must be informed of the colleague’s help, and approval must be obtained for their colleague to read the protected document. Additionally, the colleague must be identified in the confidential comments to the editor, in order to ensure that he/she is appropriately credited for any contributions ( 23 ). It is the job of the reviewer to make sure that the colleague assisting is aware of the confidentiality of the peer review process ( 23 ). Once the review is complete, the manuscript must be destroyed and cannot be saved electronically by the reviewers ( 23 ).

COMMON ERRORS IN SCIENTIFIC PAPERS

When performing a peer review, there are some common scientific errors to look out for. Most of these errors are violations of logic and common sense: these may include contradicting statements, unwarranted conclusions, suggestion of causation when there is only support for correlation, inappropriate extrapolation, circular reasoning, or pursuit of a trivial question ( 24 ). It is also common for authors to suggest that two variables are different because the effects of one variable are statistically significant while the effects of the other variable are not, rather than directly comparing the two variables ( 24 ). Authors sometimes oversee a confounding variable and do not control for it, or forget to include important details on how their experiments were controlled or the physical state of the organisms studied ( 24 ). Another common fault is the author’s failure to define terms or use words with precision, as these practices can mislead readers ( 24 ). Jargon and/or misused terms can be a serious problem in papers. Inaccurate statements about specific citations are also a common occurrence ( 24 ). Additionally, many studies produce knowledge that can be applied to areas of science outside the scope of the original study, therefore it is better for reviewers to look at the novelty of the idea, conclusions, data, and methodology, rather than scrutinize whether or not the paper answered the specific question at hand ( 24 ). Although it is important to recognize these points, when performing a review it is generally better practice for the peer reviewer to not focus on a checklist of things that could be wrong, but rather carefully identify the problems specific to each paper and continuously ask themselves if anything is missing ( 24 ). An extremely detailed description of how to conduct peer review effectively is presented in the paper How I Review an Original Scientific Article written by Frederic G. Hoppin, Jr. It can be accessed through the American Physiological Society website under the Peer Review Resources section.

CRITICISM OF PEER REVIEW

A major criticism of peer review is that there is little evidence that the process actually works, that it is actually an effective screen for good quality scientific work, and that it actually improves the quality of scientific literature. As a 2002 study published in the Journal of the American Medical Association concluded, ‘Editorial peer review, although widely used, is largely untested and its effects are uncertain’ ( 25 ). Critics also argue that peer review is not effective at detecting errors. Highlighting this point, an experiment by Godlee et al. published in the British Medical Journal (BMJ) inserted eight deliberate errors into a paper that was nearly ready for publication, and then sent the paper to 420 potential reviewers ( 7 ). Of the 420 reviewers that received the paper, 221 (53%) responded, the average number of errors spotted by reviewers was two, no reviewer spotted more than five errors, and 35 reviewers (16%) did not spot any.

Another criticism of peer review is that the process is not conducted thoroughly by scientific conferences with the goal of obtaining large numbers of submitted papers. Such conferences often accept any paper sent in, regardless of its credibility or the prevalence of errors, because the more papers they accept, the more money they can make from author registration fees ( 26 ). This misconduct was exposed in 2014 by three MIT graduate students by the names of Jeremy Stribling, Dan Aguayo and Maxwell Krohn, who developed a simple computer program called SCIgen that generates nonsense papers and presents them as scientific papers ( 26 ). Subsequently, a nonsense SCIgen paper submitted to a conference was promptly accepted. Nature recently reported that French researcher Cyril Labbé discovered that sixteen SCIgen nonsense papers had been used by the German academic publisher Springer ( 26 ). Over 100 nonsense papers generated by SCIgen were published by the US Institute of Electrical and Electronic Engineers (IEEE) ( 26 ). Both organisations have been working to remove the papers. Labbé developed a program to detect SCIgen papers and has made it freely available to ensure publishers and conference organizers do not accept nonsense work in the future. It is available at this link: http://scigendetect.on.imag.fr/main.php ( 26 ).

Additionally, peer review is often criticized for being unable to accurately detect plagiarism. However, many believe that detecting plagiarism cannot practically be included as a component of peer review. As explained by Alice Tuff, development manager at Sense About Science, ‘The vast majority of authors and reviewers think peer review should detect plagiarism (81%) but only a minority (38%) think it is capable. The academic time involved in detecting plagiarism through peer review would cause the system to grind to a halt’ ( 27 ). Publishing house Elsevier began developing electronic plagiarism tools with the help of journal editors in 2009 to help improve this issue ( 27 ).

It has also been argued that peer review has lowered research quality by limiting creativity amongst researchers. Proponents of this view claim that peer review has repressed scientists from pursuing innovative research ideas and bold research questions that have the potential to make major advances and paradigm shifts in the field, as they believe that this work will likely be rejected by their peers upon review ( 28 ). Indeed, in some cases peer review may result in rejection of innovative research, as some studies may not seem particularly strong initially, yet may be capable of yielding very interesting and useful developments when examined under different circumstances, or in the light of new information ( 28 ). Scientists that do not believe in peer review argue that the process stifles the development of ingenious ideas, and thus the release of fresh knowledge and new developments into the scientific community.

Another issue that peer review is criticized for, is that there are a limited number of people that are competent to conduct peer review compared to the vast number of papers that need reviewing. An enormous number of papers published (1.3 million papers in 23,750 journals in 2006), but the number of competent peer reviewers available could not have reviewed them all ( 29 ). Thus, people who lack the required expertise to analyze the quality of a research paper are conducting reviews, and weak papers are being accepted as a result. It is now possible to publish any paper in an obscure journal that claims to be peer-reviewed, though the paper or journal itself could be substandard ( 29 ). On a similar note, the US National Library of Medicine indexes 39 journals that specialize in alternative medicine, and though they all identify themselves as “peer-reviewed”, they rarely publish any high quality research ( 29 ). This highlights the fact that peer review of more controversial or specialized work is typically performed by people who are interested and hold similar views or opinions as the author, which can cause bias in their review. For instance, a paper on homeopathy is likely to be reviewed by fellow practicing homeopaths, and thus is likely to be accepted as credible, though other scientists may find the paper to be nonsense ( 29 ). In some cases, papers are initially published, but their credibility is challenged at a later date and they are subsequently retracted. Retraction Watch is a website dedicated to revealing papers that have been retracted after publishing, potentially due to improper peer review ( 30 ).

Additionally, despite its many positive outcomes, peer review is also criticized for being a delay to the dissemination of new knowledge into the scientific community, and as an unpaid-activity that takes scientists’ time away from activities that they would otherwise prioritize, such as research and teaching, for which they are paid ( 31 ). As described by Eva Amsen, Outreach Director for F1000Research, peer review was originally developed as a means of helping editors choose which papers to publish when journals had to limit the number of papers they could print in one issue ( 32 ). However, nowadays most journals are available online, either exclusively or in addition to print, and many journals have very limited printing runs ( 32 ). Since there are no longer page limits to journals, any good work can and should be published. Consequently, being selective for the purpose of saving space in a journal is no longer a valid excuse that peer reviewers can use to reject a paper ( 32 ). However, some reviewers have used this excuse when they have personal ulterior motives, such as getting their own research published first.

RECENT INITIATIVES TOWARDS IMPROVING PEER REVIEW

F1000Research was launched in January 2013 by Faculty of 1000 as an open access journal that immediately publishes papers (after an initial check to ensure that the paper is in fact produced by a scientist and has not been plagiarised), and then conducts transparent post-publication peer review ( 32 ). F1000Research aims to prevent delays in new science reaching the academic community that are caused by prolonged publication times ( 32 ). It also aims to make peer reviewing more fair by eliminating any anonymity, which prevents reviewers from delaying the completion of a review so they can publish their own similar work first ( 32 ). F1000Research offers completely open peer review, where everything is published, including the name of the reviewers, their review reports, and the editorial decision letters ( 32 ).

PeerJ was founded by Jason Hoyt and Peter Binfield in June 2012 as an open access, peer reviewed scholarly journal for the Biological and Medical Sciences ( 33 ). PeerJ selects articles to publish based only on scientific and methodological soundness, not on subjective determinants of ‘impact ’, ‘novelty’ or ‘interest’ ( 34 ). It works on a “lifetime publishing plan” model which charges scientists for publishing plans that give them lifetime rights to publish with PeerJ, rather than charging them per publication ( 34 ). PeerJ also encourages open peer review, and authors are given the option to post the full peer review history of their submission with their published article ( 34 ). PeerJ also offers a pre-print review service called PeerJ Pre-prints, in which paper drafts are reviewed before being sent to PeerJ to publish ( 34 ).

Rubriq is an independent peer review service designed by Shashi Mudunuri and Keith Collier to improve the peer review system ( 35 ). Rubriq is intended to decrease redundancy in the peer review process so that the time lost in redundant reviewing can be put back into research ( 35 ). According to Keith Collier, over 15 million hours are lost each year to redundant peer review, as papers get rejected from one journal and are subsequently submitted to a less prestigious journal where they are reviewed again ( 35 ). Authors often have to submit their manuscript to multiple journals, and are often rejected multiple times before they find the right match. This process could take months or even years ( 35 ). Rubriq makes peer review portable in order to help authors choose the journal that is best suited for their manuscript from the beginning, thus reducing the time before their paper is published ( 35 ). Rubriq operates under an author-pay model, in which the author pays a fee and their manuscript undergoes double-blind peer review by three expert academic reviewers using a standardized scorecard ( 35 ). The majority of the author’s fee goes towards a reviewer honorarium ( 35 ). The papers are also screened for plagiarism using iThenticate ( 35 ). Once the manuscript has been reviewed by the three experts, the most appropriate journal for submission is determined based on the topic and quality of the paper ( 35 ). The paper is returned to the author in 1-2 weeks with the Rubriq Report ( 35 ). The author can then submit their paper to the suggested journal with the Rubriq Report attached. The Rubriq Report will give the journal editors a much stronger incentive to consider the paper as it shows that three experts have recommended the paper to them ( 35 ). Rubriq also has its benefits for reviewers; the Rubriq scorecard gives structure to the peer review process, and thus makes it consistent and efficient, which decreases time and stress for the reviewer. Reviewers also receive feedback on their reviews and most significantly, they are compensated for their time ( 35 ). Journals also benefit, as they receive pre-screened papers, reducing the number of papers sent to their own reviewers, which often end up rejected ( 35 ). This can reduce reviewer fatigue, and allow only higher-quality articles to be sent to their peer reviewers ( 35 ).

According to Eva Amsen, peer review and scientific publishing are moving in a new direction, in which all papers will be posted online, and a post-publication peer review will take place that is independent of specific journal criteria and solely focused on improving paper quality ( 32 ). Journals will then choose papers that they find relevant based on the peer reviews and publish those papers as a collection ( 32 ). In this process, peer review and individual journals are uncoupled ( 32 ). In Keith Collier’s opinion, post-publication peer review is likely to become more prevalent as a complement to pre-publication peer review, but not as a replacement ( 35 ). Post-publication peer review will not serve to identify errors and fraud but will provide an additional measurement of impact ( 35 ). Collier also believes that as journals and publishers consolidate into larger systems, there will be stronger potential for “cascading” and shared peer review ( 35 ).

CONCLUDING REMARKS

Peer review has become fundamental in assisting editors in selecting credible, high quality, novel and interesting research papers to publish in scientific journals and to ensure the correction of any errors or issues present in submitted papers. Though the peer review process still has some flaws and deficiencies, a more suitable screening method for scientific papers has not yet been proposed or developed. Researchers have begun and must continue to look for means of addressing the current issues with peer review to ensure that it is a full-proof system that ensures only quality research papers are released into the scientific community.

This paper is in the following e-collection/theme issue:

Published on 17.4.2024 in Vol 26 (2024)

This is a member publication of National University of Singapore

Comparing Open-Access Database and Traditional Intensive Care Studies Using Machine Learning: Bibliometric Analysis Study

Authors of this article:

Original Paper

• Yuhe Ke 1 * , MBBS   ; 
• Rui Yang 2 * , MSc   ; 
• Nan Liu 2 , PhD  

1 Division of Anesthesiology and Perioperative Medicine, Singapore General Hospital, Singapore, Singapore

2 Centre for Quantitative Medicine, Duke-NUS Medical School, National University of Singapore, Singapore, Singapore

*these authors contributed equally

Corresponding Author:

Nan Liu, PhD

Centre for Quantitative Medicine

Duke-NUS Medical School

National University of Singapore

8 College Road

Singapore, 169857

Phone: 65 66016503

Email: [email protected]

Background: Intensive care research has predominantly relied on conventional methods like randomized controlled trials. However, the increasing popularity of open-access, free databases in the past decade has opened new avenues for research, offering fresh insights. Leveraging machine learning (ML) techniques enables the analysis of trends in a vast number of studies.

Objective: This study aims to conduct a comprehensive bibliometric analysis using ML to compare trends and research topics in traditional intensive care unit (ICU) studies and those done with open-access databases (OADs).

Methods: We used ML for the analysis of publications in the Web of Science database in this study. Articles were categorized into “OAD” and “traditional intensive care” (TIC) studies. OAD studies were included in the Medical Information Mart for Intensive Care (MIMIC), eICU Collaborative Research Database (eICU-CRD), Amsterdam University Medical Centers Database (AmsterdamUMCdb), High Time Resolution ICU Dataset (HiRID), and Pediatric Intensive Care database. TIC studies included all other intensive care studies. Uniform manifold approximation and projection was used to visualize the corpus distribution. The BERTopic technique was used to generate 30 topic-unique identification numbers and to categorize topics into 22 topic families.

Results: A total of 227,893 records were extracted. After exclusions, 145,426 articles were identified as TIC and 1301 articles as OAD studies. TIC studies experienced exponential growth over the last 2 decades, culminating in a peak of 16,378 articles in 2021, while OAD studies demonstrated a consistent upsurge since 2018. Sepsis, ventilation-related research, and pediatric intensive care were the most frequently discussed topics. TIC studies exhibited broader coverage than OAD studies, suggesting a more extensive research scope.

Conclusions: This study analyzed ICU research, providing valuable insights from a large number of publications. OAD studies complement TIC studies, focusing on predictive modeling, while TIC studies capture essential qualitative information. Integrating both approaches in a complementary manner is the future direction for ICU research. Additionally, natural language processing techniques offer a transformative alternative for literature review and bibliometric analysis.

Introduction

The start of critical care as a medical subspecialty can be traced back to a polio epidemic during which a substantial number of patients needed prolonged mechanical ventilation [ 1 ]. Over time, the field of critical care has experienced significant growth and continual evolution. Research in this field has played a pivotal role in unraveling the complexities of numerous diseases and treatment modalities, driving substantial advancements in clinical practice over the past decades [ 2 ]. Groundbreaking studies have investigated critical areas such as sepsis, mechanical ventilation, acute lung and kidney injuries, intensive care unit (ICU) delirium, and sedation in critically ill patients [ 3 ].

These research studies have often been conducted in traditional ways such as prospective and randomized controlled trials [ 4 ], cohort and observational studies, clinical trials [ 5 ], and clinical and translational research [ 6 ]. These traditional methods have revolutionized patient care and improved outcomes significantly. For instance, the implementation of protocol-driven, goal-directed management of sepsis and appropriate fluid therapy has led to remarkable reductions in mortality rates [ 7 , 8 ], and these findings have been integral in developing evidence-based practice guidelines that are now the gold standard [ 9 , 10 ].

Despite their undeniable merits, traditional research methods in intensive care also come with several limitations [ 11 ]. Clinical trials are known for their high costs [ 12 ], stringent standardization requirements, and ethical oversight [ 13 ]. Data collection can be laborious, prone to human errors, and constrained in terms of quantity and granularity [ 14 ]. Moreover, obtaining patient consent for most randomized controlled trials in the ICU poses challenges [ 15 ], necessitating alternative consent models. These limitations have become increasingly apparent as medical complexity continues to grow exponentially [ 16 ].

The advent of electronic health records (EHRs) has heralded a new era in clinical research by facilitating the digitization of health care systems [ 17 ]. In this era of data science, a more integrated approach can be adopted, using machine learning (ML) algorithms to tackle the complexity of critical illness [ 18 ]. Open-access databases (OADs), such as the Medical Information Mart for Intensive Care (MIMIC) database [ 19 ] and the Philips eICU Collaborative Research Database (eICU-CRD) [ 20 ], have played a transformative role by enabling free data sharing.

The concept of free and open databases plays a pivotal role in promoting data sharing and advancing medical knowledge in accordance with the findable, accessible, interoperable, and reusable (FAIR) guiding principle. The FAIR principles, which emphasize that data should be findable, accessible, interoperable, and reusable, are essential for fostering a collaborative and transparent scientific research environment [ 21 , 22 ]. By removing barriers to access, free, and open databases allow researchers, regardless of their affiliations or resources, to contribute to and benefit from the collective pool of information. Accessibility fosters inclusivity and diversity in research, promoting a broader range of perspectives and approaches to medical challenges. This democratization of knowledge leads to a more equitable distribution of information. Researchers can now leverage these vast repositories of information for ML and artificial intelligence studies, marking a departure from traditional intensive care (TIC) research approaches.

Conducting a literature review [ 23 ] to investigate the disparities between traditional ICU research and studies based on open-access data sets holds significant importance as it provides a comprehensive understanding of the strengths and limitations of the latter. However, conventional methods of literature reviews and bibliometric analysis have their limitations, especially when dealing with large-scale literature due to computational complexity and the labor-intensive nature of manual interpretations [ 24 - 26 ]. To address these challenges, natural language processing (NLP) offers a promising avenue, while topic modeling techniques can be used to extract various topic themes from extensive data sets [ 27 , 28 ].

Built on the foundations of bidirectional encoder representations from transformers (BERT), BERTopic introduces a novel approach to topic modeling [ 29 , 30 ]. Unlike traditional unsupervised models like latent Dirichlet allocation, which rely on “bag-of-words” model [ 31 ], BERTopic overcomes the problem of semantic information loss, significantly enhancing the accuracy of generated topics, and providing more interpretable compositions for each topic, which greatly facilitates the classification of topics.

With the aid of BERTopic, this study aims to shed light on the disparities and commonalities between studies conducted through OADs and TIC research. By analyzing the overall trends and patterns in these 2 groups, we seek to identify knowledge gaps and explore avenues for complementary contributions between these research approaches.

Data Filtering

We performed an ML-based analysis of research abstracts in the Web of Science (WoS) database to automatically categorize the research papers to conduct this literature mapping analysis. There was no limit to the year of publication of the articles. The search query consisted of the following keyword to identify all the studies that were published under the umbrella of intensive care: (“ICU” OR “intensive care”). The search terms were deliberately left to be broad to cover broad spectrums of journals in the field.

The inclusion criteria were as follows: (1) written in English, (2) articles that had keywords related to intensive care, (3) articles that had the article type of “article” or “review.” We excluded articles with incomplete data fields (eg, title, abstract, publication year, and paper citation). The articles included were then further processed to identify if they were studies using OADs. These articles were labeled as “open-access database,” while the rest of the articles extracted were labeled as “traditional intensive care.”

The search used for this study was performed on January 18, 2023, from WoS. This generated 227,893 search results, which were subsequently reselected using Python. An advanced search from PubMed was done based on the broad search terms of ICU studies used from previous Cochrane ICU literature review [ 32 ] to ensure the accuracy of the results. The numbers corroborated with a discrepancy of 4.9% (227,893 WoS keyword search vs 239,748 PubMed ICU keyword search).

Selection Criteria for OADs

A title search using keywords from all currently existing OADs was conducted to identify OAD studies. These include (1) MIMIC [ 19 ], (2) eICU-CRD [ 20 ], (3) Amsterdam University Medical Centers Database (AmsterdamUMCdb) [ 33 ], (4) High Time Resolution ICU Dataset (HiRID) [ 34 ], and (5) Pediatric Intensive Care database [ 35 ]. We avoided including only keywords in the search and restricted the search years by the year that the OAD was made publicly available to reduce the inadvertent inclusion of incorrect articles due to keywords. For instance, the search term for OADs published with the MIMIC database included title keyword search with the terms (“MIMIC-IV” OR “MIMIC-III” OR “MIMIC-II” OR “MIMIC Dataset” OR “medical information mart for intensive care” OR “MIMIC IV” OR “MIMIC III” OR “MIMIC II”) in studies that were published after 2003. The title keyword search for the searches and the year of cutoff for each OAD are presented in Multimedia Appendix 1 .

Furthermore, to ensure the accuracy of the supervised keyword classification, a manual review of the classification by 2 critical care physicians was done for 100 articles from each category that were randomly selected. The review was done independently with the physicians labeling the extract publications into OAD and TICs. An accuracy of 99% was achieved on independent reviews, and full agreement was achieved after discussion on the discrepancy. The final results were matched with the supervised keyword classification.

We performed a bibliometric analysis by directly extracting publication details from the WoS database using Python (Python Software Foundation). The analysis involved assessing the number of articles published per year, calculating total citation counts, and identifying the top journals that published intensive care-related articles. Comprehensive results are presented in Multimedia Appendix 2 .

Data Analysis

Uniform manifold approximation and projection.

Uniform manifold approximation and projection (UMAP) is a manifold learning technique for dimension reduction, which can identify key structures in high-dimensional data space and map them to low-dimensional space to accomplish dimensionality reduction. Compared to other dimensionality reduction algorithms, such as principal component analysis [ 36 ], UMAP can retain more global features [ 37 ]. In this paper, we constructed a corpus consisting of abstract words from all studies. However, due to the massive size of the corpus, visualizing and analyzing the high-dimensional data to explore the differences in the vocabulary patterns between the OAD and TIC studies is a challenge. The UMAP package in Python, which implements the UMAP algorithm, was used to project the high-dimensional corpus to 4 dimensions. By cross plotting each dimension, we were able to investigate underlying differences in corpus distribution between OAD and TIC studies.

Topic modeling can help us explore the similarities and differences between research topics in OAD and TIC studies. Unlike conventional topic modeling models, BERTopic uses the BERT framework for embeddings, enabling a deeper understanding of semantic relationships [ 30 ]. The BERTopic model was implemented by the BERTopic package in Python and divided 146,727 studies into 30 topic IDs. We also performed latent Dirichlet allocation topic modeling through Python’s LdaModel package for comparison. Through the review of topic keywords by 2 critical care physicians, BERTopic exhibited superior accuracy and sophistication in topic identification, with enhanced interpretability and scientific rigor.

Consequently, the BERTopic model was used for the final analysis. Each of these topics was given a corresponding clinical research category. The overlapping categories were merged into topic families for easier comparisons. By using these advanced techniques, we were able to uncover hidden patterns and relationships within the literature and provide insights into the current state of intensive care research.

A total of 227,893 records were identified from the WoS database on January 18, 2023, of which 195,463 full records were subsequently processed. Records were excluded if they are not “article” or “review” or if they do not contain keywords related to intensive care. After exclusions, 145,426 articles were identified as TIC studies and 1301 articles were categorized as OAD ( Figure 1 ).

We examined the number of articles published per year to analyze the trends in TIC and OAD studies ( Figure 2 ). Over the past 2 decades, TIC studies have experienced exponential growth, culminating in a peak of 16,378 articles in 2021. A subsequent decline in the number of publications occurred in 2022, likely attributable to delayed indexing within the WoS database and a reduction in COVID-19–related studies as the pandemic stabilized [ 38 ]. In contrast, the first OAD study emerged in 2003, with its popularity experiencing a consistent upsurge since 2018. Nonetheless, the number of OAD publications remains markedly lower in comparison to TIC publications.

The OAD studies were published most frequently in new open-access journals such as Frontiers in Medicine , Frontiers in Cardiovascular Medicine , and Scientific Reports while the TIC studies were published most frequently in established journals like Critical Care Medicine , Intensive Care Medicine , and Critical Care ( Multimedia Appendix 2 ). Further analysis of keywords from the abstracts showed 2.4% (3492/145,426) TIC studies were meta-analyses or systematic reviews, while only 0.08% (1/1301) OAD study was in this category. There were 5.61% (73/1301) OAD studies, and 7.43% (10,799/145,426) TIC studies that had the keyword of “cost.” Examples of the data fields that are available within OADs such as MIMIC and eICU-CRD are listed in Textbox 1 . Some information fields such as end-of-life goals and values and health care provider psychology are not available within the current EHRs extracted for OADs.

Examples of information that is available in current OADs

• Patient information: demographics and social set-up
• Hospital context: admission time and discharge time, intensive care unit (ICU) and hospital admissions, and pre-ICU admission
• Diagnosis: physician-curated ICU diagnosis and data-driven phenotypes
• Intervention: medications, procedures, and organ support
• Diagnostics: blood test, microbiology, and scans
• Clinical texts: clinical notes and diagnostic reports
• Physiological monitoring: basic monitoring and waveforms

Examples of information that is not readily available in current OADs

• Patient information: family set up and visiting, financial information, and special populations
• Hospital context: post-ICU discharge details, delayed admission or discharge, and health personnel psychology
• Diagnosis: pre-ICU history and diagnosis requiring clinical symptoms
• Intervention: indications for interventions, complications, and intraoperative and postoperative
• Diagnostics: pathology photographs, imaging, and molecular or genetic studies
• Clinical texts: patient narratives, end-of-life goals and patient value, and health personnel behavior
• Physiological monitoring: advanced monitoring

The UMAP algorithm was used to project the high-dimension corpus to 4 dimensions and allowed exploration of the vocabulary patterns between the OAD and TIC studies ( Figure 3 ). The projection values are represented by the x-axis, while the densities are represented by the y-axis. The presence of considerable overlap between TIC studies and OAD studies suggests that they share a substantial number of common terminologies, which may correspond to similar research topics. Nonetheless, TIC studies exhibit a more extensive coverage than OAD studies, which may stem from broader research scope and extended research duration.

Subsequently, the BERTopic model was then used to generate 30 topic IDs ( Figure 4 ). The internal commonalities of each topic ID were reviewed by critical care physicians and assigned a specific subtopic in intensive care research. The model was able to automatically classify the topics with high interpretability and the topic components were interpreted with ease. For instance, components in topic ID 5 consist of, in decreasing order of weightage: “learning,” “model,” “machine,” “machine learning,” “models,” “data,” “prediction,” and “performance.” This topic was consequently labeled “predictive model” (topic ID 5 in Multimedia Appendix 3 ).

The overall topic distribution in TIC studies was more uniform, while the OAD studies tended to be concentrated on several topics including topic ID 2 (kidney injury), 5 (predictive model), and 13 (sepsis). Some topics that were missing in OAD studies included 6 (pediatrics care), 21 (viral infections), 23 (health personnel and psychology), and 28 (nutrition and rehabilitation).

The similarity matrix shows that there was little overlap between the topics ( Multimedia Appendix 4 ). To facilitate the interpretability of the categories, the overlapping topic IDs were merged to form the final 22 topic families ( Multimedia Appendix 3 ).

Topics such as “healthcare associated infection,” “thoracic surgeries,” and “pregnancy related” research were among the more frequently discussed 15 topics in TIC studies but have limited publications in OAD studies. The topics of “predictive model,” “obesity,” and “fungal infections” were popular in OAD studies but not the TIC studies. Overall, the topic distributions of the TIC studies were distributed more evenly with the topic family of sepsis accounting for a quarter of the studies, while publications in the OAD studies were heavily skewed toward the predictive model (>40%) and sepsis (>30%; Figure 5 ).

Principal Results

This study conducted a comprehensive review and bibliometric analysis of OAD and TIC studies. NLP was used to facilitate this large-scale literature review. Studies using OADs mainly concentrated on a few topics, such as predictive modeling, while TIC studies covered a wider range of topics with a more balanced distribution.

Advantages of OAD Studies

OAD studies offer several advantages that have contributed to their increasing popularity in intensive care research. The granularity of data and easy access to large-cohort databases, such as MIMIC [ 39 ], has enabled researchers to perform predictive modeling and conduct various secondary analyses efficiently [ 40 , 41 ]. This accessibility has provided valuable opportunities for exploring specific aspects of patient care, evident in studies investigating phenomena like “weekend effects” and circadian rhythms in ICU patients before discharge [ 42 - 46 ]. The vast amount of longitudinal and time series data available in OADs has also facilitated the implementation of complex ML and deep learning methods [ 47 ].

Limitations of OAD Studies

However, it is crucial to acknowledge the retrospective nature of OAD data, which inherently limits the assessment of confounding factors and the ability to draw strong causal conclusions. The observational design of OAD studies may result in lower-quality evidence according to the GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) framework [ 48 , 49 ], and thus, the research from OAD studies has yet to be fully integrated into existing evidence-based guidelines, as exemplified by the omission of OAD studies in the 2021 sepsis guidelines [ 50 ]. Nevertheless, OADs remain a valuable resource for supplementing and complementing TIC studies, providing unique insights and enhanced predictive scores for intensive care settings.

Furthermore, approximately 50% of the studies using OADs published focused on predictive modeling. The increased usage of ML methods in predictive modeling has not been without critique. Some medical prediction problems inherently possess linear characteristics, and the selection of features may predominantly focus on already known strong predictors, leading to limited improvements in prediction accuracy with ML [ 51 ]. Additionally, interstudy heterogeneity poses a challenge in comparing results obtained from different ML models applied to the same data sets [ 52 ]. The ethical implications of relying solely on ML models to make high-risk health care decisions instead of involving clinical expertise are also relevant considerations [ 51 , 53 ].

While OADs provide comprehensive patient data, there are certain limitations in their ability to capture specific information essential for certain critical care research areas. Notably, data fields related to qualitative aspects such as ethics and end-of-life care [ 54 , 55 ], and health care personnel psychology [ 56 ] may be challenging, if not impossible, to obtain through OADs generated from EHRs. Consequently, TIC studies have played a pivotal role in addressing these limitations by capturing critical information that is integral to understanding ethical considerations, patient experiences, and health care provider psychology in intensive care [ 57 , 58 ].

Synergy Between OAD and TIC Studies

The synergy between OAD and TIC studies is a promising approach to enhance the comprehensiveness and robustness of intensive care research. OADs, with their large cohort sizes, can serve as external validation cohorts for ML models developed from TIC studies, potentially reducing the sample sizes required for prospective research. Furthermore, OAD studies can corroborate the results of TIC studies, benefiting from larger sample sizes and real-world data, thus providing more practical insights for implementation in intensive care settings [ 43 ]. The integration of OAD and TIC studies presents an opportunity to bridge the gaps in data availability and research methodologies, ultimately enriching the understanding and practice of critical care medicine.

Potential Impact of NLP

The usage of large language models such as BERTopic has proven to be a valuable tool for large-scale literature review and topic extraction [ 58 ]. This approach has enabled accurate, reliable, and granular topic generation, offering clinicians a more effective means of interpreting data compared to traditional bag-of-words models [ 59 ]. The potential of NLP to analyze scientific articles and identify trends and knowledge gaps holds promise for shaping the future of research in critical care medicine. As the volume of publications in critical care continues to grow and large language modeling continues to advance in health care [ 60 ], AI technology will be crucial in efficiently identifying and predicting emerging trends.

Future Directions

Future research in the field of critical care can explore novel applications of ML beyond predictive modeling. For instance, using ML to study patterns in how papers are cited, shared, and discussed on the web could help predict their potential impact on the scientific community. This analysis can aid in identifying highly influential papers and understanding the factors that contribute to their recognition. Additionally, investigations into methods for enhancing the interpretability and transparency of ML algorithms in critical care research would further facilitate the ethical and responsible use of AI technologies.

Strengths and Limitations

The study’s application of NLP-driven in analyzing scientific articles and identifying trends highlights the potential impact of AI technologies in streamlining literature reviews and identifying emerging trends more efficiently.

Another notable strength of this study is the usage of the WoS database, the world’s oldest and most extensively used repository of research publications and citations, encompassing approximately 34,000 journals [ 61 ]. The comprehensiveness of this database provides a robust representation of the literature in the field of intensive care research. Nevertheless, it is essential to acknowledge that some articles published in nonindexed journals might not have been captured, and future studies could benefit from considering additional databases to supplement our findings.

One other limitation lies in the classification of OAD and TIC studies, which may be subject to variations in the interpretation of keywords. However, we optimized the keyword combinations during the search process in the WoS database and implemented Python filtering techniques, resulting in a relatively high level of accuracy in our classifications. The number of studies was further corroborated with a manual search on PubMed and a review of the classifications of the studies was done by critical care physicians.

Although there were no specific language restrictions, the nature of the search term being in English inadvertently excluded valuable contributions from non-English research. This may potentially limit the generalizability of our findings to a broader international audience. In future investigations, the inclusion of articles from various languages could offer a more comprehensive and diverse perspective on intensive care research.

Conclusions

This study has provided valuable insights into the expanding landscape of intensive care research through a comprehensive bibliometric analysis of a large number of publications by leveraging NLP technologies. While OAD studies have demonstrated significant promise, it is essential to view them as a complementary approach rather than a replacement for TIC studies. The unique strength of TIC studies lies in their ability to capture crucial qualitative information, which is essential for comprehensive and ethical decision-making. The integration of both OAD and TIC studies offers a synergistic approach to enriching our understanding of critical care medicine and advancing patient care outcomes. As NLP technology continues to advance, it holds the potential to offer a feasible and transformative alternative for literature review and bibliometric analysis.

Acknowledgments

We thank Dr Nicholas Brian Shannon for assistance with the manual review of the supervised keyword classification. This work was supported by the Duke-NUS Signature Research Programme, funded by the Ministry of Health, Singapore.

Data Availability

The data sets generated during and/or analyzed during this study are available from the corresponding author on reasonable request. The complete set of code used in this study is readily available for download on GitHub [ 62 ].

Authors' Contributions

YK and NL played key roles in the conceptualization of the project. RY was responsible for formalizing the methodology and conducting data curation with the advisory of YK. YK contributed to the validation of the data, ensuring its relevance to the research objectives. RY took the lead in visualizing the data. Both YK and RY drafted the original manuscript. NL served as the project supervisor, overseeing the implementation, and providing valuable input in the writing, review, and editing phases.

Conflicts of Interest

None declared.

Search terms for open-access database (OAD) studies with the cutoff by the years of publications.

Top 20 journals ranked by total citation in which the open-access database and traditional intensive care studies were published. The average citation per article was obtained with the total citation/total number of articles. The citation counts were obtained from Web of Science.

Topic ID and topic family and the components and weightage in each of the categories.

Similarity matrix of 30 topics.

• Kelly FE, Fong K, Hirsch N, Nolan JP. Intensive care medicine is 60 years old: the history and future of the intensive care unit. Clin Med (Lond). 2014;14(4):376-379. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cook D, Brower R, Cooper J, Brochard L, Vincent JL. Multicenter clinical research in adult critical care. Crit Care Med. 2002;30(7):1636-1643. [ CrossRef ] [ Medline ]
• Rosenberg AL, Tripathi RS, Blum J. The most influential articles in critical care medicine. J Crit Care. 2010;25(1):157-170. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Granholm A, Alhazzani W, Derde LPG, Angus DC, Zampieri FG, Hammond NE, et al. Randomised clinical trials in critical care: past, present and future. Intensive Care Med. 2022;48(2):164-178. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Markey KA, Ottridge R, Mitchell JL, Rick C, Woolley R, Ives N, et al. Assessing the efficacy and safety of an 11β-hydroxysteroid dehydrogenase type 1 inhibitor (AZD4017) in the idiopathic intracranial hypertension drug trial, IIH:DT: clinical methods and design for a phase II randomized controlled trial. JMIR Res Protoc. 2017;6(9):e181. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Verdonk F, Feyaerts D, Badenes R, Bastarache JA, Bouglé A, Ely W, et al. Upcoming and urgent challenges in critical care research based on COVID-19 pandemic experience. Anaesth Crit Care Pain Med. 2022;41(5):101121. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gurnani PK, Patel GP, Crank CW, Vais D, Lateef O, Akimov S, et al. Impact of the implementation of a sepsis protocol for the management of fluid-refractory septic shock: a single-center, before-and-after study. Clin Ther. 2010;32(7):1285-1293. [ CrossRef ] [ Medline ]
• Wang JL, Chin CS, Chang MC, Yi CY, Shih SJ, Hsu JY, et al. Key process indicators of mortality in the implementation of protocol-driven therapy for severe sepsis. J Formos Med Assoc. 2009;108(10):778-787. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Levy MM, Evans LE, Rhodes A. The surviving sepsis campaign bundle: 2018 update. Crit Care Med. 2018;46(6):997-1000. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fan E, Del Sorbo L, Goligher EC, Hodgson CL, Munshi L, Walkey AJ, et al. An official American Thoracic Society/European Society of Intensive Care Medicine/Society of Critical Care Medicine clinical practice guideline: mechanical ventilation in adult patients with acute respiratory distress syndrome. Am J Respir Crit Care Med. 2017;195(9):1253-1263. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Goldfrad C, Vella K, Bion JF, Rowan KM, Black NA. Research priorities in critical care medicine in the UK. Intensive Care Med. 2000;26(10):1480-1488. [ CrossRef ] [ Medline ]
• Moore TJ, Heyward J, Anderson G, Alexander GC. Variation in the estimated costs of pivotal clinical benefit trials supporting the US approval of new therapeutic agents, 2015-2017: a cross-sectional study. BMJ Open. 2020;10(6):e038863. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Umscheid CA, Margolis DJ, Grossman CE. Key concepts of clinical trials: a narrative review. Postgrad Med. 2011;123(5):194-204. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Maré IA, Kramer B, Hazelhurst S, Nhlapho MD, Zent R, Harris PA, et al. Electronic data capture system (REDCap) for health care research and training in a resource-constrained environment: technology adoption case study. JMIR Med Inform. 2022;10(8):e33402. [ FREE Full text ] [ CrossRef ] [ Medline ]
• O'Hearn K, Gibson J, Krewulak K, Porteous R, Saigle V, Sampson M, et al. Consent models in Canadian critical care randomized controlled trials: a scoping review. Can J Anaesth. 2022;69(4):513-526. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ghassemi M, Celi LA, Stone DJ. State of the art review: the data revolution in critical care. Crit Care. 2015;19(1):118. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. Big data in health care: using analytics to identify and manage high-risk and high-cost patients. Health Aff (Millwood). 2014;33(7):1123-1131. [ CrossRef ] [ Medline ]
• Mlodzinski E, Wardi G, Viglione C, Nemati S, Crotty Alexander L, Malhotra A. Assessing barriers to implementation of machine learning and artificial intelligence-based tools in critical care: web-based survey study. JMIR Perioper Med. 2023;6:e41056. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Johnson AEW, Bulgarelli L, Shen L, Gayles A, Shammout A, Horng S, et al. MIMIC-IV, a freely accessible electronic health record dataset. Sci Data. 2023;10(1):1. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pollard TJ, Johnson AEW, Raffa JD, Celi LA, Mark RG, Badawi O. The eICU collaborative research database, a freely available multi-center database for critical care research. Sci Data. 2018;5:180178. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Inau ET, Sack J, Waltemath D, Zeleke AA. Initiatives, concepts, and implementation practices of FAIR (findable, accessible, interoperable, and reusable) data principles in health data stewardship practice: protocol for a scoping review. JMIR Res Protoc. 2021;10(2):e22505. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wilkinson MD, Dumontier M, Aalbersberg IJJ, Appleton G, Axton M, Baak A, et al. The FAIR guiding principles for scientific data management and stewardship. Sci Data. 2016;3:160018. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bahadoran Z, Mirmiran P, Kashfi K, Ghasemi A. Importance of systematic reviews and meta-analyses of animal studies: challenges for animal-to-human translation. J Am Assoc Lab Anim Sci. 2020;59(5):469-477. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Haidich AB. Meta-analysis in medical research. Hippokratia. 2010;14(Suppl 1):29-37. [ FREE Full text ] [ Medline ]
• Thompson DF, Walker CK. A descriptive and historical review of bibliometrics with applications to medical sciences. Pharmacotherapy. 2015;35(6):551-559. [ CrossRef ] [ Medline ]
• Donthu N, Kumar S, Mukherjee D, Pandey N, Lim WM. How to conduct a bibliometric analysis: an overview and guidelines. J Bus Res. 2021;133:285-296. [ FREE Full text ] [ CrossRef ]
• Zhao W, Chen JJ, Perkins R, Liu Z, Ge W, Ding Y, et al. A heuristic approach to determine an appropriate number of topics in topic modeling. BMC Bioinformatics. 2015;16(Suppl 13):S8. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Doanvo A, Qian X, Ramjee D, Piontkivska H, Desai A, Majumder M. Machine learning maps research needs in COVID-19 literature. Patterns (N Y). 2020;1(9):100123. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Devlin J, Chang MW, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. ArXiv. Preprint posted online on May 24 2019. 2018. [ CrossRef ]
• Grootendorst M. BERTopic: neural topic modeling with a class-based TF-IDF procedure. ArXiv. Preprint posted online on March 11 2022. 2022. [ CrossRef ]
• Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Mach Learn Res. 2003;3:993-1022. [ FREE Full text ]
• Delaney A, Bagshaw SM, Ferland A, Manns B, Laupland KB, Doig CJ. A systematic evaluation of the quality of meta-analyses in the critical care literature. Crit Care. 2005;9(5):R575-R582. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Thoral PJ, Peppink JM, Driessen RH, Sijbrands EJG, Kompanje EJO, Kaplan L, et al. Sharing ICU patient data responsibly under the Society of Critical Care Medicine/European Society of Intensive Care Medicine joint data science collaboration: the Amsterdam university medical centers database (AmsterdamUMCdb) example. Crit Care Med. 2021;49(6):e563-e577. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Faltys M, Zimmermann M, Lyu X, Hüser M, Hyland S, Rätsch G, et al. HiRID, a high time-resolution ICU dataset. PhysioNet. 2021. URL: https://physionet.org/content/hirid/1.1.1/ [accessed 2024-04-02]
• Zeng X, Yu G, Lu Y, Tan L, Wu X, Shi S, et al. PIC, a paediatric-specific intensive care database. Sci Data. 2020;7(1):14. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Maćkiewicz A, Ratajczak W. Principal components analysis (PCA). Comput Geosci. 1993;19(3):303-342. [ CrossRef ]
• McInnes L, Healy J, Melville J. UMAP: uniform manifold approximation and projection for dimension reduction. ArXiv. Preprint posted online on September 18 2020. 2018. [ CrossRef ]
• Murray CJL. COVID-19 will continue but the end of the pandemic is near. Lancet. 2022;399(10323):417-419. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mark R. The story of MIMIC. In: Secondary Analysis of Electronic Health Records. Cham, Switzerland. Springer International Publishing; 2016;43-49.
• Alghatani K, Ammar N, Rezgui A, Shaban-Nejad A. Predicting intensive care unit length of stay and mortality using patient vital signs: machine learning model development and validation. JMIR Med Inform. 2021;9(5):e21347. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liu D, Zheng M, Sepulveda NA. Using artificial neural network condensation to facilitate adaptation of machine learning in medical settings by reducing computational burden: model design and evaluation study. JMIR Form Res. 2021;5(12):e20767. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhang Z, Ho KM, Hong Y. Machine learning for the prediction of volume responsiveness in patients with oliguric acute kidney injury in critical care. Crit Care. 2019;23(1):112. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Meyer A, Zverinski D, Pfahringer B, Kempfert J, Kuehne T, Sündermann SH, et al. Machine learning for real-time prediction of complications in critical care: a retrospective study. Lancet Respir Med. 2018;6(12):905-914. [ CrossRef ] [ Medline ]
• Chen H, Zhu Z, Zhao C, Guo Y, Chen D, Wei Y, et al. Central venous pressure measurement is associated with improved outcomes in septic patients: an analysis of the MIMIC-III database. Crit Care. 2020;24(1):433. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Faust L, Feldman K, Chawla NV. Examining the weekend effect across ICU performance metrics. Crit Care. 2019;23(1):207. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Davidson S, Villarroel M, Harford M, Finnegan E, Jorge J, Young D, et al. Vital-sign circadian rhythms in patients prior to discharge from an ICU: a retrospective observational analysis of routinely recorded physiological data. Crit Care. 2020;24(1):181. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Xie F, Yuan H, Ning Y, Ong MEH, Feng M, Hsu W, et al. Deep learning for temporal data representation in electronic health records: a systematic review of challenges and methodologies. J Biomed Inform. 2022;126:103980. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ, et al. GRADE Working Group. What is "quality of evidence" and why is it important to clinicians? BMJ. 2008;336(7651):995-998. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Evans L, Rhodes A, Alhazzani W, Antonelli M, Coopersmith CM, French C, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Crit Care Med. 2021;49(11):e1063-e1143. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Volovici V, Syn NL, Ercole A, Zhao JJ, Liu N. Steps to avoid overuse and misuse of machine learning in clinical research. Nat Med. 2022;28(10):1996-1999. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fleuren LM, Klausch TLT, Zwager CL, Schoonmade LJ, Guo T, Roggeveen LF, et al. Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Med. 2020;46(3):383-400. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yoon CH, Torrance R, Scheinerman N. Machine learning in medicine: should the pursuit of enhanced interpretability be abandoned? J Med Ethics. 2022;48(9):581-585. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gillett GR. Intensive care unit research ethics and trials on unconscious patients. Anaesth Intensive Care. 2015;43(3):309-312. [ CrossRef ] [ Medline ]
• Aulisio MP, Chaitin E, Arnold RM. Ethics and palliative care consultation in the intensive care unit. Crit Care Clin. 2004;20(3):505-523, x-xi. [ CrossRef ] [ Medline ]
• Raudenská J, Steinerová V, Javůrková A, Urits I, Kaye AD, Viswanath O, et al. Occupational burnout syndrome and post-traumatic stress among healthcare professionals during the novel coronavirus disease 2019 (COVID-19) pandemic. Best Pract Res Clin Anaesthesiol. 2020;34(3):553-560. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Davidson JE, Jones C, Bienvenu OJ. Family response to critical illness: postintensive care syndrome-family. Crit Care Med. 2012;40(2):618-624. [ CrossRef ] [ Medline ]
• White DB, Angus DC, Shields AM, Buddadhumaruk P, Pidro C, Paner C, et al. A randomized trial of a family-support intervention in intensive care units. N Engl J Med. 2018;378(25):2365-2375. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Popoff B, Occhiali É, Grangé S, Bergis A, Carpentier D, Tamion F, et al. Trends in major intensive care medicine journals: a machine learning approach. J Crit Care. 2022;72:154163. [ CrossRef ] [ Medline ]
• Yang R, Tan TF, Lu W, Thirunavukarasu AJ, Ting DSW, Liu N. Large language models in health care: development, applications, and challenges. Health Care Sci. 2023;2(4):255-263. [ FREE Full text ] [ CrossRef ]
• Birkle C, Pendlebury DA, Schnell J, Adams J. Web of science as a data source for research on scientific and scholarly activity. Quant Sci Stud. 2020;1(1):363-376. [ FREE Full text ] [ CrossRef ]
• GitHub. URL: https://github.com/YangRui525/Comparing-OAD-and-TIC-Studies

Abbreviations

Edited by A Mavragani; submitted 19.04.23; peer-reviewed by D Chrimes, S Pesälä; comments to author 14.07.23; revised version received 01.08.23; accepted 14.01.24; published 17.04.24.

©Yuhe Ke, Rui Yang, Nan Liu. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 17.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

The Scholarly Kitchen

What’s Hot and Cooking In Scholarly Publishing

Honest Signaling and Research Integrity

• Data Publishing
• Metrics and Analytics
• Research Integrity

Editor’s Note: Today’s post is coauthored by Tim Vines and Ben Kaube. Ben is a cofounder of Cassyni, a platform for helping publishers and institutions create and engage communities of researchers using seminars.

Assessing research integrity issues for articles submitted to an academic journal is a daunting prospect: the journal must somehow use the submitted text, figures, and (if available) accompanying data to uncover unethical behavior by the authors. This behavior (such as fabricating or altering data, etc.) may have taken place long before the article itself was written. 

However, these methods primarily focus on detecting bad behavior. To complement this approach, there’s a compelling need for indicators of “honest signaling” in research — a concept that, instead of aiming to catch bad behavior, captures commitment to ethical research practices.

It is instructive to draw analogy to the evolutionary biology concept known as “honest signaling.” This principle illustrates how certain traits in the animal kingdom, such as the peacock’s extravagant tail or the bright colors of some jumping spiders, serve as ‘honest’ markers of the individual’s fitness . These traits are energetically costly to produce and maintain, signifying that only individuals in prime condition can afford these displays. Thus, these signals are a reliable indicator of an individual’s fitness and hence their quality as a mate.

Translating this concept to the realm of research integrity, we seek analogous signals within researcher behavior — practices that are sufficiently demanding of time, effort, or resources, such that they are unlikely to be undertaken by those not genuinely committed to ethical research. Honest signaling in this context would encompass activities that are both visible and verifiable, shedding light on the researcher’s dedication to transparency and integrity:

• Open Science Practices: This includes the sharing of raw data, pre-registration of studies, publishing of preprints, and sharing of the code and scripts to operate command line software. These practices not only require a significant investment of time and resources but also open the researcher’s work to scrutiny and verification by the wider community.
• Community Engagement: Active participation in the scientific community through presentations at conferences, seminars, and poster sessions acts as a form of honest signaling. Such engagements are not only resource-intensive, but also expose one’s research to critique and validation by peers. Moreover, maintaining a visible online presence (e.g., personal webpages and ORCID profiles) and contributions to journal publishing outside of authorship roles (e.g., writing peer reviews and membership on editorial boards) further exemplifies a researcher’s commitment to open discourse and collaboration.

These signals, akin to the peacock’s tail, are not just markers of the quality and credibility of the individual’s work but also serve to signal their commitment to the principles of research integrity.

Any indicators of honest signaling should be considered contextually, acknowledging that open research practices differ between fields and engagement with scholarly communities varies regionally. This is particularly true when considering researchers facing systemic barriers such as those in the Global South. This suggests that simple scores of honest signaling will be overly reductive and that honest signaling indicators should be evaluated qualitatively as part of the editorial process.

The continued growth of digital platforms and the increase in online data post-COVID offer new opportunities to operationalize honest signaling indicators. By aggregating data from society meetings, departmental web pages, and conference programs, it becomes possible to capture engagement of researchers within their respective fields at scale and produce a suite of indicators for research integrity purposes (e.g., what is the publication record of author X? have they presented similar work at relevant conferences or seminar series? Does the manuscript have accompanying data or code?).

One important consideration when aggregating data from many venues is the difficulty of author disambiguation. ORCID has an important role to play here , but the usual challenge of preventing false positives is compounded by the fact that many research outputs associated with honest signaling have traditionally lacked persistent identifiers and high quality metadata. For example posters and seminar recordings have only recently started receiving persistent identifiers. 

In biology, honest signals are an indicator of an individual’s fitness, and while some of those signals – like the peacock’s tail – might be eye-catching to us, they are fine tuned to be evaluated by individuals in the same species. Similarly, in the world of research, it will always be the experts in one’s own field that will be best positioned to pick up on and assess honest signaling.

The concept of honest signaling offers a promising framework for verifying research integrity in a manner that complements existing tools aimed at detecting misconduct. By focusing on behaviors that are costly in terms of time and effort and indicate a researcher’s commitment to ethical practices, this approach not only helps identify instances of integrity but also encourages a culture of openness and engagement within the scientific community. Operationalizing indicators of “honest signaling”, while challenging, presents an opportunity to foster transparency and accountability in academic research, ensuring that it remains a pursuit marked by ethical conduct and genuine discovery.

Tim Vines is the Founder and Project Lead on DataSeer, an AI-based tool that helps authors, journals and other stakeholders with sharing research data. He's also a consultant with Origin Editorial, where he advises journals and publishers on peer review. Prior to that he founded Axios Review, an independent peer review company that helped authors find journals that wanted their paper. He was the Managing Editor for the journal Molecular Ecology for eight years, where he led their adoption of data sharing and numerous other initiatives. He has also published research papers on peer review, data sharing, and reproducibility (including one that was covered by Vanity Fair). He has a PhD in evolutionary ecology from the University of Edinburgh and now lives in Vancouver, Canada.

Ben Kaube is a cofounder of Cassyni, a platform for helping publishers and institutions create and engage communities of researchers using seminars.

17 Thoughts on "Honest Signaling and Research Integrity"

Tim, you provide an interesting thought piece, but I think that you’ve pushed “honest signaling” way past its original meaning and conflated several practical and operational issues of journal publishing into the amorphous concept of “honesty.”

First, the concept of “honest signaling” in biology is contestable, because one needs to ascribe intent behind the action. Is the peacock being “honest” when displaying his tail, or just showing his sexual fitness? And what about all of the examples of mimicry and false signals in the biological world? Is a Viceroy butterfly being “dishonest” by mimicking the pattern of the toxic monarch butterfly so it won’t be a bird’s lunch or is it just being strategic?

Second, you list a number of things that authors can do to increase “honest signals” in publishing, such as depositing datasets, preprints, posters, etc. “Transparency” would be a better, more accurate term here than the amorphous construct of “honesty.” Similarly, we could use the term “longevity” to describe how many papers have been published previously by an author; “peer validation” to describe how colleagues view this author; “merit” to describe the institutions (labs, universities, funders) that vouch for that individual, etc. Honesty is not a yes/no checkbox–I’m not accusing you of writing this–but the logic of your data collection leads an evaluator to make an honesty evaluation.

Like citations or alt-metrics, I fear that your intent on collating these “honesty signals” and using them in the manuscript decision-making process will be counter-productive. Eugene Garfield kept arguing until his death that citations and Impact Factor scores should not be used alone and only in context. Look how far honesty got him.

• By Phil Davis
• Apr 16, 2024, 9:23 AM
• Reply to Comment

Comments crossed, but I’m with Phil that running with the ‘honest signaling’ concept from ecology and evolutionary biology doesn’t go far, since there are probably more examples of deception, and deception works. Still, if we think of “Transparency” instead of “honest signaling” I don’t follow the rationale that “collating these ‘honesty signals’ and using them in the manuscript decision-making process will be counter-productive.” Because anything can be gamed and careful fakery is hard to detect? I don’t see that as discounting the value of transparency in the rigor of studies.

• By Chris Mebane
• Apr 16, 2024, 10:01 AM

The counter-productive comment was about where the argument was heading–creating an “honesty score” or metric that would be used by a triage editor in their decision-making.

Most authors are first authors, so they don’t have any history. And many senior (last) authors would get a very high honesty score on a manuscript but may have little to do the with the integrity of the manuscript, and may not have even read it. Honesty signals just confuses and obfuscates the real details that you want. These can be dealt with by check-listing all the details that you want with a manuscript (data deposit, institutional email address, contribution statement, etc.).

I realize that this cuts out any AI or machine-learning algorithms that a startup company sells to publishers to scour the web, gather and score the “honesty” of any author. Honesty is not an easy construct to operationalize, and personally, I think the outcome of such an effort would be worse than what we currently have. Call me pragmatic.

• Apr 16, 2024, 10:49 AM

Hi Phil – you may find this concept easier to digest if you view the ‘individual’ here as the research article, rather than the researcher themselves. The manuscript as submitted is the dull brown bird: it’s very hard to judge whether the authors are just going through the motions to get another publication, or whether they’re committed to contributing to knowledge.

The manuscript equivalent of the ‘peacock’s tail’ are the associated outputs for that piece of research (the datasets, code objects, conference posters, presentations, etc); as these take considerable extra work to produce and also allow readers to inspect the ‘internal state’ of the manuscript. For example, a reader opening the data file and seeing that it’s a disordered mess can conclude that the manuscript is likely flawed. Putting in the effort to make a poster about the research and taking it to a conference shows that the authors want scrutiny for their work.

• By Tim Vines
• Apr 16, 2024, 1:26 PM

Just read the manuscript.

• By Mike Rossner
• Apr 16, 2024, 1:28 PM

Deceptive adaptations are indeed very common in nature – but all the examples you give relate to deceiving other species. Deceiving potential mates about your quality (or lack thereof) is a valid evolutionary strategy*, but this is exactly why mate choice has evolved to focus on traits that are hard to fake. Sure, you’re a good dancer, but first run a marathon & we’ll dance at the finish line.

(*yes, we know evolution doesn’t have intent, but it takes twice as many words to describe it correctly & it’s also boring for the reader)

• Apr 16, 2024, 1:43 PM

Nice post. Could adequate methods detail such as presence of RRIDs and a reasonable explanation of the limitations of the study not be a quality signal?

As you know these are nontrivial additions to the paper, that signal quality and integrity. Cell line RRID inclusion is associated with a reduction in the use of problematic cell lines as we showed in 2019 (Babic paper).

• By Anita Bandrowski
• Apr 16, 2024, 9:35 AM

Getting RRIDs for all of the reagents & software in the manuscript is a significant amount of work, so authors that put in this extra effort – particularly when the journal doesn’t require it – should get some recognition.

• Apr 16, 2024, 11:43 AM

Excellent essay. I must admit I liked it in part because it fits my view that research integrity and reliability is enhanced through greater transparency, not by hiding honest signaling or the lack thereof behind double-blind reviewing. Your essay seems to target editorial offices to look for honest signaling, but I think a large part also falls to the peer reviewers. The argument for blinding reviewers to signaling, honest or otherwise, is that blindness prevents status bias wherein manuscripts from Prof. Big Name from Elite University are welcomed in the Journal of High Impact Factors where author No Name from Southern Latitude University is not. I’m not sure what the best antidote is for the status bias issue, but I don’t think hiding honest signaling from reviewers is it.

• Apr 16, 2024, 9:41 AM

I know this is meant well but it reads like a protocol for being a good Soviet citizen to me. I wonder how many Nobel prizes were won by good boys and girls. In a wider publishing context it comes across like not publishing JK Rowling for her personal views. We are publishing research articles not judging the authors

• By Nick Weir-Williams
• Apr 16, 2024, 10:39 AM

Strongly agree. The data in each article need to be evaluated for their merits and integrity. There are no work arounds.

• Apr 16, 2024, 11:18 AM

The authors don’t seem to be arguing that papers lacking these signals shouldn’t be published. But it is reasonable that readers should view a research article in which all the data are online differently from one in which not a scrap of data was shared. The data is the evidence – it’s what makes science reproducible and falsifiable. Nullius in verba, right? Similarly, a researcher who issues a pre-registration is holding themselves accountable to a stated hypothesis and research plan. One who does not is free to move the goal posts surreptitiously as many times as they want. The difference here really matters in science, and we should notice it.

• By Michele Avissar-Whiting
• Apr 17, 2024, 10:39 AM

Great article. I love your perspective. I think you’ve (re)coined a powerful concept for Research Integrity. You draw your parallels from biology, while I draw my parallels from the financial industry.

Online credit card fraud has similar integrity issues. They are solving these issues with Know Your Customer (KYC) controls and “honesty signals” as to whether an individual transaction is genuine or fraudulent.

As you suggest, I don’t believe you can fight AI with AI, in some kind of AI arm’s race. If we’ve learnt anything from the Ancient Greeks, it’s that if you want to beat an opponent one most focus on the Achilles heel. In the case of paper mills, bad actors and fabricated results, that’s truth and reality… and as you have so eloquently put it… honesty signals.

• By Paul Killoran
• Apr 16, 2024, 11:06 AM

Thanks for this interesting piece, Tim and Ben. I am not sure the concept of “honest signaling” is a perfect analogy for assessing the integrity of a researcher or research output, particularly if there is an implication that additional work might need to be performed to create these signals beyond the good-faith work that honest researchers undertake anyway. To me, this has similarities with the wasteful effort that is caused by the “proof of work” algorithms used by some block chain algorithms.

However, I am not sure that you are suggesting this. Perhaps, you are instead proposing that the work that honest researchers do anyway through, for example, following open science practices and engaging fully in their communities, can act as positive signals of their integrity if shared transparently, accessibly and in a verifiable way. This aligns very much with the work that we are doing at ORCID to encourage the sharing and re-use of what we term “trust markers”, those elements of an ORCID record that constitute positive signals of a researcher’s contributions over time.

In addition to the core elements of publications and other research outputs, education and affiliations, and funding awards, it is already possible to include acts of professional service in ORCID profiles. Such activities can include participation in boards and committees, membership of standards bodies and expert panels, the organization of and participation in conferences, and of course membership of editorial boards. Like all data in ORCID records, subject to the privacy controls of the researcher, this data is available for inspection and re-use by anyone who might find it useful, and always includes provenance information (i.e. who made the assertion and when).

To make it as easy as possible for researchers to share this information about themselves, we encourage the conveners of such activities, who are the authorities on who is participating, to contribute that data directly to ORCID (always with the researcher’s permission). By doing so, organizations join what we call the Community Trust Network, increasing the overall transparency of research activities, allowing maximum re-use of this information, all while without unduly burdening the researcher with additional work to maintain and share this data manually.

• By Chris Shillum, ORCID
• Apr 16, 2024, 2:37 PM

Hi Chris – we’re definitely focused on the activities that diligent researchers do as part of conducting and communicating their work. It’s great that ORCID is working to systematically collect and expose these.

• Apr 17, 2024, 10:11 AM

Kudos to the authors — this was really well articulated and the analogies were cute and helpful! [start shameless plug] It’s also a great complement to this SKitch article right here: https://scholarlykitchen.sspnet.org/2021/06/02/guest-post-the-10000-watt-bulb-how-preprints-shine-a-light-on-misconduct/ [end shameless plug].

We know instinctively that transparency and deception are anticorrelated, but the dominant research communication processes and incentive systems favor opacity.

Call them “honest markers”, “trust signals”, whatever…we need more of them and more incentives for their use. Happily, most of them have the added benefit of making the research more reproducible!

• Apr 16, 2024, 10:37 PM

Quibbles about evolutionary biology aside, this article makes an important point. Naively trusting submissions because of words and graphs on a page, or even an author name, leads to negative outcomes, especially as human- and machine-agent productions are becoming more difficult to distinguish on a daily basis. Hard to fake, multimodal, and transparent open science practices are an important part of the emerging discipline of forensic scientometrics, and publishers need to become adept at this quickly if they are to remain relevant in the rapidly evolving open science ecosystems.

• By Mark G. Bilby
• Apr 17, 2024, 8:41 AM

Leave a Comment Cancel reply

Notify me of follow-up comments by email.

Related Articles:

Next Article: