can chatgpt write summary of research paper

'ZDNET Recommends': What exactly does it mean?

ZDNET's recommendations are based on many hours of testing, research, and comparison shopping. We gather data from the best available sources, including vendor and retailer listings as well as other relevant and independent reviews sites. And we pore over customer reviews to find out what matters to real people who already own and use the products and services we’re assessing.

When you click through from our site to a retailer and buy a product or service, we may earn affiliate commissions. This helps support our work, but does not affect what we cover or how, and it does not affect the price you pay. Neither ZDNET nor the author are compensated for these independent reviews. Indeed, we follow strict guidelines that ensure our editorial content is never influenced by advertisers.

ZDNET's editorial team writes on behalf of you, our reader. Our goal is to deliver the most accurate information and the most knowledgeable advice possible in order to help you make smarter buying decisions on tech gear and a wide array of products and services. Our editors thoroughly review and fact-check every article to ensure that our content meets the highest standards. If we have made an error or published misleading information, we will correct or clarify the article. If you see inaccuracies in our content, please report the mistake via this form .

How to use ChatGPT to summarize a book, article, or research paper

AI chatbots like ChatGPT can be used to make summarizing long articles, research papers, and books an easier job. If you're tasked with writing a summary for school or work about a body of written text, and you're pinched for time, ChatGPT can help you understand the necessary components.

You should remember that ChatGPT is a tool that can help you further understand a topic, and it may not be in your best interest to have it write your work for you.

Also:   How to make ChatGPT provide sources and citations

If you're a student writing a research paper, someone who is keen to discover more about a lengthy article, or someone who wants to dive into a complicated subject, you can use ChatGPT to simplify the process.

How ChatGPT can create summaries for you

Materials needed : You'll need a device that can connect to the internet, an OpenAI account , and a basic understanding of the article, research paper, or book you want to summarize. 

Also:  This AI chatbot can sum up any PDF and answer any question you have about it

The process should take about one to three minutes.

1. Find your article, paper, or book to summarize

If you need ChatGPT to help summarize an article or research paper, find the body of text online and keep it open in a separate tab.

2. Open your web browser and log in to OpenAI

Open your web browser of choice and type in chat.openai.com/chat . 

Also: How to use ChatGPT to write Excel formulas

Log in or sign up for an account. 

3. Type your request

In the chat box, type in TLDR: followed by the title of the book you want summarized. TLDR stands for too long, didn't read. 

Also: How to use ChatGPT: Everything you need to know

You can also ask ChatGPT: "Summarize [book title]." 

You cannot paste a URL into ChatGPT. If you do so, you'll get a response explaining that the chatbot cannot access websites. If you want ChatGPT to summarize research, you'll need a basic understanding of the topic to ensure you aren't including inaccurate information in your report.

For example, if you're writing a paper about how twin sibling dynamics affect their marriages, you can ask ChatGPT: "Explain how twin relationships affect their marriages."

Also:   The best AI chatbots: ChatGPT and other noteworthy alternatives 

ChatGPT will offer you a few bullet points of both positive and negative effects a twin bond can have on the twins' marriages. You'll still need to read key points of the research, as ChatGPT won't provide you with specific statistics or scientific conclusions. But ChatGPT can help you understand the context surrounding the research you're interested in.

If you're having trouble comprehending specific passages in an article, book, or research paper, you can copy parts of the text and paste them into ChatGPT.

Think of this method as reshuffling the words you're currently reading to help you make more sense of the text in front of you.

This is an accurate summary of 'The Hunger Games: Mockingjay'.

What are ChatGPT's limitations?

If you're using ChatGPT to summarize an article, book, or piece of research, keep in mind that ChatGPT isn't aware of events that occurred after September 2021. 

Also:   4 things Claude AI can do that ChatGPT can't

ChatGPT is a large language model that uses queues and millions of data points to mimic human responses. This form of mimicry is why ChatGPT will answer questions even when it doesn't output the correct answer. So, make sure you're not using any information from ChatGPT without fact-checking it.

If you try to get around this obstacle and provide ChatGPT with an article that contains information post-2021, it might hallucinate. Here, I asked the chatbot to summarize an article about a new app I wrote about , and it made up a few details. 

Lemon8 is a new app from TikTok's parent company, ByteDance. Although the TikTok trend may exist, that's not what the article is about.

Can ChatGPT summarize a PDF?

Copying and pasting a URL into ChatGPT won't yield you the best results. If there is a specific paragraph or sentence you're struggling with, you can copy and paste it into ChatGPT and ask it to explain it to you. Still, it's best to read the PDF and use the chatbot as a summary tool and not as an educator. 

Also:  How to use ChatGPT to write code

If you're looking for an AI chatbot that you can regularly rely on to give you an accurate summary of a PDF, consider using ChatPDF . You can summarize up to three PDFs of up to 120 pages per day, and an upgraded plan is available for $5 per month.

Can ChatGPT summarize an email thread?

Sort of. If you want to copy and paste every single email, ChatGPT can summarize the thread's contents for you. It would be more helpful to scan an email thread yourself and ask ChatGPT to help you write a response based on the key points you know about the conversation. 

Editor's note: We've added additional context to the step concerning ChatGPT summarizing articles by URL. 

More on AI tools

How to make chatgpt provide sources and citations, how to use chatgpt, how to use chatgpt to write a cover letter (and why you should).

Productivity

How to use ChatGPT to summarize an article

Full disclosure: ChatGPT's ability to summarize articles—or any other text, really—is hit or miss. But I've spent many hours modifying my prompts and experimenting with different article lengths to test the limits of what it can ( and can't ) do.   

If you're looking to use ChatGPT to summarize an article, book, or research paper, here's how. 

Table of contents:

How to write an effective ChatGPT article summary prompt

Limitations of summarizing articles with ChatGPT  

How to use ChatGPT to summarize an article: FAQs  

Here's the short version of how to use ChatGPT to summarize any text (but keep reading for a deeper dive): 

Find and copy your digital source material. 

Log in to chat.openai.com .

Paste the text in the message box on the ChatGPT home page, and ask ChatGPT to summarize the text. 

Or, if you have ChatGPT Plus :

Find a digital article to summarize and copy the URL.

Select Model > Web Browsing .

Paste the URL of the source material in the message box, and ask ChatGPT to summarize the contents of the link.

Now for the details.

Find an article to summarize

Find the body of text you want ChatGPT to summarize, and keep it open in a separate tab. You'll need this later. 

I'm using this episode recap on the series finale of The Good Place by Vulture. (Warning: spoilers ahead!) 

Log in to OpenAI 

Go to chat.openai.com , and log in to your account.

Don't have an OpenAI account yet? Here's how to set one up . 

Ask ChatGPT to summarize an article 

ChatGPT can't directly browse the internet (unless you're a paid subscriber), so you'll need to feed it with the raw material it needs before asking it to generate a summary. 

Copy the relevant text of your digital source material. 

Paste the text in the message bar of ChatGPT's home page, add " Summarize the article above " to the end of your query, and hit Enter . 

If you're a ChatGPT Plus subscriber, there's no need for the copy-and-paste step. Instead, switch to the Web Browsing model ( Model > Web Browsing ), and then paste the URL of your source material into the message bar and ask ChatGPT to summarize it. 

If you're not satisfied with the response, you can click Regenerate response to get a ChatGPT to try again. Or you can modify your prompt. 

Note : ChatGPT Plus subscribers used to have the ability to prompt ChatGPT to browse the web, but the feature was disabled by OpenAI due to it displaying content in ways that weren't authorized. The company says they're working to bring it back in the future.

Tip: Need to extract text from webpages to use in ChatGPT? You can try a free tool like this , which converts HTML—the standard markup language for webpages—to plain text.

When it comes to getting ChatGPT to give you the article summary you actually want, here are two key ways to refine your prompt . 

1. Tell it how long the response should be 

To ensure the summary is short and sweet, add a character limit or a range of acceptable lengths to your prompt. For example,  "The summary should be 125 characters max" (this is equivalent to roughly a half page of text, double-spaced, using 12-point font) or "Write a 150–300-word summary of this article." 

Sometimes, it's helpful to include the desired length of ChatGPT's response in your initial prompt. Other times, it's not. (Remember how I said ChatGPT's ability to summarize text is hit or miss?) Here's an example from the first time I asked it to summarize the final episode of The Good Place using a maximum of 125 characters. 

Technically, ChatGPT did exactly what I asked it to do—but it took "summarize" to the extreme. In this case, I modified the prompt and removed the character limit, which gave me a slightly better response. (But there aren't nearly enough specifics for my liking.) 

At the end of the day, you'll have to play around with what details work for each prompt. 

2. Specify what to include in your article summary

A clear prompt with sufficient detail is key to getting the response you want from ChatGPT. 

Let's say I want to know what happens to each main character in the series finale of The Good Place. For that, I'd prompt ChatGPT with, "Try again, but tell me what happens to each main character."

Now that's the kind of spoiler summary I'm looking for. 

Limitations of summarizing articles with ChatGPT 

ChatGPT is still fairly new, so there are a couple of things you should keep in mind when using it to summarize articles. 

Token limits . GPT-3 and -3.5—the free GPT model available to anyone—has a limit of 4,096 tokens , which is equal to roughly 3,000 words. (The more advanced GPT-4 model offers nearly double the amount.) For most news articles, the token limit shouldn't be a concern. But you'll likely run into issues if you ask ChatGPT to summarize a lengthy research paper or a novel. 

"Plausible-sounding but incorrect" responses . ChatGPT's been known to produce less-than-accurate answers , so take its responses with a grain of salt (read: fact check, fact check, fact check). 

Oversimplification. Since ChatGPT lacks critical-thinking skills, it has a tendency to strip away important details and subtle nuances presented in the source material, leaving you with a less-than-complete understanding. For something trivial like getting summaries of spoilers for pure entertainment, this isn't a huge concern. But if you're using ChatGPT to summarize an article that you'll reference in, say, a college dissertation, I strongly recommend reading the article in its entirety so you don't miss out on crucial details or nuance.

How to use ChatGPT to summarize an article: FAQs 

OpenAI is constantly updating existing features and adding new functionality, so it's understandable if you're confused about what ChatGPT can and can't do. Here are answers to the most common questions about how to summarize with ChatGPT.

Can ChatGPT summarize a pdf?

ChatGPT can summarize a PDF and answer questions about the PDF, but this feature is available only to ChatGPT Plus subscribers. 

It's worth noting, though, that in my experience, the quality of ChatGPT's responses varies depending on the length, complexity, and formatting of the file. For example, when I asked ChatGPT to summarize a PDF that contained headings, it generated a formatted list of key talking points along with a one-sentence summary of each point. When I asked ChatGPT to summarize a PDF with the same content, but no headings, it generated only a vague, one-paragraph summary. 

Can ChatGPT summarize videos?

ChatGPT doesn't have the functionality to watch or analyze video content. There are workarounds, though. For example, if you have access to the video transcript, you can upload that into ChatGPT and ask it to analyze the transcript. Or, if you're a ChatGPT Plus or Enterprise user, there are custom GPTs you can use to summarize videos. 

Can ChatGPT summarize a website?

Yes, but there are limitations. If you ask ChatGPT to summarize a website containing content that requires a subscription—for example, an article from the New York Times —ChatGPT won't be able to summarize it. This feature is also available only to paid users. 

Automate your AI article summaries 

You can take the manual process out of summarizing articles you come across on the web. With OpenAI and Zapier, you can automatically generate AI summaries in just two clicks—directly from your browser. Click on the Zap template below to get started. 

Create article summaries with OpenAI from the Zapier Chrome extension

To get started with a Zap template—what we call our pre-made workflows—just click on the button. It only takes a few minutes to set up. You can read more about setting up Zaps here.

Related reading:

6 ways to use the Zapier ChatGPT plugin

How to automatically generate email summaries with Zapier and OpenAI

ChatGPT vs. Bard: What's the difference?

How to integrate ChatGPT with Microsoft Outlook

How to train ChatGPT to sound like you

Get productivity tips delivered straight to your inbox

We’ll email you 1-3 times per week—and never share your information.

Jessica Lau

Jessica Lau is a senior content specialist at Zapier. Outside of writing, she likes to snuggle her dogs, and provide unsolicited podcast and book recommendations.

• Artificial intelligence (AI)

Related articles

How to add an email signature in Outlook

How to create and customize Airtable views

Why won't Google Docs let me paste without installing an extension? 

Why won't Google Docs let me paste without...

How to use IMPORTRANGE in Google Sheets

Improve your productivity automatically. Use Zapier to get your apps working together.

Home → Academic Writing → How to Use ChatGPT to Write a Research Paper: Tips and Tricks to Get You Started

How to Use ChatGPT to Write a Research Paper: Tips and Tricks to Get You Started

Jordan Kruszynski

• January 4, 2024

If you’re an academic, you’ve probably spent a lot of time planning and writing research papers. It’s a fine art, and one that requires a fair amount of skill, precision and preparation. But whether you’re an old master in the field, or a rookie embarking on your first serious research paper, there are tools that can help you to simplify the planning stages and organise your thoughts more effectively.

One of those tools is the (in)famous ChatGPT , and it’s this that we’ll be focusing on in our article. In the right hands, ChatGPT can become a powerful research tool that will make your paper-writing that little bit easier. So sit back, relax, and discover our tips and tricks for using ChatGPT to write a research paper.

What is ChatGPT?

Just in case you don’t already know, ChatGPT is an artificial intelligence tool developed by OpenAI that can help you with your research. It uses natural language processing to understand what you’re looking for and provide you with relevant information. You can ask it questions, and it will provide you with answers in a conversational style, as well as offer sources to back up its information.

One of the biggest advantages of ChatGPT is that it can save you time. Instead of spending hours searching for sources, you can simply ask ChatGPT for help. This can bring you a reliable list of sources for further investigation fairly quickly. It’s crucial to note however that the AI shouldn’t be exploited to do the actual writing of the paper for you. This could see you accused of plagiarism or misconduct, and besides, as a researcher, you’re probably rightfully proud of your ability to write a compelling paper.

Another advantage of ChatGPT is that it’s always available (even when libraries or other sources of information might be inaccessible) so you can work on your research paper at any time of day or night.

Interested in learning more about how AI programs like ChatGPT are changing the academic landscape? Listen to Oxford researcher Samantha-Kaye Johnston’s views from the frontline in this exciting episode of The Research Beat podcast.

The benefits of using ChatGPT to write a research paper

There are many benefits to using ChatGPT for research papers. Firstly, as we mentioned earlier, it can save you time . A slow drag of several hours looking for specific sources can be reduced to just a few minutes with the AI’s help.

Secondly, it can help you find sources that you might not have found otherwise . ChatGPT has access to a wide range of sources, including academic journals and books.

Thirdly, it can help you organise your research . ChatGPT can provide you with a summary of the information you’ve gathered, making it easier to analyse and integrate into your research paper.

How to use ChatGPT to write a research paper – a step by step guide

• Start by creating a list of questions that you want to answer in your research paper.
• Open ChatGPT and ask it one of the questions on your list, for example, ‘What is the critical history of feminist literature in Europe?’
• ChatGPT will provide you with a list of sources to check out.
• Read through the sources and take notes on the information that is relevant to your research question.
• Repeat steps 2-4 for each question on your list.
• Once you’ve gathered all of your information, organise it into an outline for your research paper.
• Use the information you’ve gathered to write your research paper.

Working with your sources

Once you have your sources in order, you might want to use prompts to get help from ChatGPT with other parts of the writing process. A prompt is a specific instruction to the AI that can give you tailored information or responses. For example, if you’re struggling to understand part of another research paper, you could use the following prompt:

‘Please explain the following paragraphs in simple words. I am having trouble understanding (insert concept here).’

Input the prompt along with the relevant passage from the source, and ChatGPT will provide a summary that could help you to unlock your understanding of the tricky concept.

Looking for ChatGPT prompts tailor-made for academics? Check out Audemic’s list of over 50 prompts to help you with your work and research!

Writing tips and tricks for using ChatGPT

When using ChatGPT to write a research paper, it’s important to keep a few things in mind. Firstly, make sure that you’re using reliable sources . ChatGPT can provide you with a list of sources, but it’s up to you to determine which ones are reliable.

Secondly, make sure that you’re paraphrasing the information you’ve gathered in your own words . You don’t want to cheat or be accused of it.

Finally, make sure that you’re using the information you’ve gathered to answer your research questions . Everything you uncover through ChatGPT should be used to feed your own understanding and improve the quality and precision of your answers.

Common mistakes to avoid when using ChatGPT to write a research paper

While ChatGPT is an excellent tool for research papers, there are some common mistakes to avoid:

• Crucially, don’t rely too heavily on ChatGPT. It’s essential to do some of your research on your own and use ChatGPT to supplement it.
• Don’t forget to cite your sources correctly. Just because ChatGPT provided you with the information doesn’t mean that you don’t need to cite it. Moreover, ChatGPT cannot actually produce academic citations for you.
• Always remember to proofread your research paper carefully, especially if you’ve used AI elements to construct it.

ChatGPT vs. traditional research methods

While traditional research methods have their advantages and always will, ChatGPT, as we’ve seen, has some of its own. We think one of the best uses for AI programs like ChatGPT is to accelerate parts of the paper-writing process that would otherwise take hours. If you can use the AI to produce a list of interesting and relevant sources, then you can get to work quickly as an academic, studying and analysing those sources to determine their value within your paper. In general, if you approach ChatGPT with an attitude of maintaining quality and integrity, then it can only enhance your work.

Final Thoughts

ChatGPT is everywhere at the moment, and while it has stirred up a great deal of controversy thanks to its implications for academic integrity, it can be an excellent tool for helping to write research papers . The key is using it correctly, and not relying too heavily on it. Focusing on how it can enhance your already-sharp academic writing skills will allow you to save time, find plenty of valuable sources, and organise your paper’s structure more effectively. And that’s how to use ChatGPT to write a research paper!

ChatGPT is not the only AI tool that can help with these parts of the paper-writing puzzle. If you want to listen to academic papers, break them down into digestible pieces and freely take notes on them, then Audemic is waiting for you. Try it for free today!

Keep striving, researchers! ✨

Table of Contents

Related articles.

How to Publish a Research Paper: A Step-by-Step Guide

You’re in academia. You’re going steady. Your research is going well and you begin to wonder: ‘How exactly do I get a

Behind the Scenes: What Does a Research Assistant Do?

Have you ever wondered what goes on behind the scenes in a research lab? Does it involve acting out the whims of

How to Write a Research Paper Introduction: Hook, Line, and Sinker

Want to know how to write a research paper introduction that dazzles? Struggling to hook your reader in with your opening sentences?

Blog Podcast

Privacy policy Terms of service

Subscribe to our newsletter!

Discover more from Audemic: Access any academic research via audio

Subscribe now to keep reading and get access to the full archive.

Type your email…

Continue reading

How to use ChatGPT to summarize an article

Save time when you know how to use ChatGPT to summarize an article

Knowing how to use ChatGPT to summarize an article is useful when you’re in a rush and looking for the key points of an article. You might be a fast reader, but no one can compete with an AI. It can also often help to understand more complicated subject matter if it’s presented in smaller chunks. Of course, it’s always worth going back and reading the article properly when you have more time, to make sure you get the full gist of it. 

We know you would never summarize one of our lovingly-written articles on Tom’s Guide, but for other sites and sources, here’s how to use ChatGPT to summarize an article. 

And we’ll keep it brief, we promise. 

How to use ChatGPT to summarize an article  

• Log in and select the chat bar
• Type TLDR and link to the article
• Press send Read on to see detailed instructions for each step.

As of the time of writing the main ways to use ChatGPT to summarize an article are on the new Bing with ChatGPT (which you may not have access to yet) or on OpenAI ’s own website , where you can make a free account and then sign in. One thing to bear in mind is the openai.com version of the chatbot is limited to information pre-September 2021. Both methods use the same command, TLDR, which is internet speak for “Too long, didn’t read”. Make sure to put this before the text you wish to summarize.  

How to use ChatGPT to summarize an article - on ChatGPT.com  

1. log in and select the chat bar.

Login to https://chat.openai.com and select the chat bar at the bottom of the page. 

2. Type TLDR and link to the article

Type in TLDR and then paste a link to the article you wish to summarize.

3. Press send

Select the send button (or press enter) and then wait for ChatGPT's response. Rest assured, the chatbot can skim the article much quicker than you can.

How to use Bing with ChatGPT to summarize an article 

• Select chat
• Enter TLDR and the article link
• Press enter and wait Read on to see detailed instructions for each step.

1. Select chat

Navigate to the Bing homepage in your browser of choice, select chat .

2. Enter TLDR and the article link

Enter TLDR and then copy and paste the link to the article that you want summed up.

3. Press enter and wait

Press enter and then wait for Bing to prepare its response.

There you go, remember of course that just because an AI has summarized an article, it won't have altered the information at all. That means if the information isn't accurate to begin with, it won't be accurate in the summary. Your best bet is to stick to trusted sources like Tom’s Guide!

If you want some more AI assistance why not check out the 7 best ChatGPT tips to get the most out of the chatbot , how to use ChatGPT for travel advice or how to use the Dall-E 2 AI image generator . 

Sign up to get the BEST of Tom’s Guide direct to your inbox.

Upgrade your life with a daily dose of the biggest tech news, lifestyle hacks and our curated analysis. Be the first to know about cutting-edge gadgets and the hottest deals.

Andy is Tom’s Guide’s Trainee Writer, which means that he currently writes about pretty much everything we cover. He has previously worked in copywriting and content writing both freelance and for a leading business magazine. His interests include gaming, music and sports- particularly Formula One, football and badminton. Andy’s degree is in Creative Writing and he enjoys writing his own screenplays and submitting them to competitions in an attempt to justify three years of studying.  

OpenAI confirms ChatGPT event for Monday — 'feels like magic'

OpenAI could drop its own search engine on Monday — here’s what it needs to take on Google

Lomachenko vs Kambosos Jr live stream: How to watch boxing online, fight card, start time, odds

Most Popular

• 2 Apple iPhone 16 Pro tipped to get this major display upgrade
• 3 Apple could unveil revamped Siri with generative AI at WWDC — here’s the key upgrades
• 4 Plush queen mattress vs firm queen mattress: Which is best for your budget and sleep?
• 5 How to watch Google I/O 2024: Android 15, Gemini and more

How to Use ChatGPT for Summarization: A Step-by-Step Guide

In today's world, we are bombarded with more information than we can possibly handle. We don't have the time or the energy to read everything that comes across our screens. That's where summarization comes in. Summarization is the process of distilling large amounts of information into a shorter, more manageable form. It can be a lifesaver when you need to quickly understand the key points of a long document, article, or report.

Understanding ChatGPT and Summarization

Before we dive into the details of how to use ChatGPT for summarization, it's important to first understand what ChatGPT is and why summarization is important.

ChatGPT is an open-source natural language processing (NLP) technology developed by OpenAI. It is designed to generate text that is indistinguishable from text written by humans. ChatGPT is a state-of-the-art language model that is capable of understanding and generating human-like language. It has been trained on a massive dataset of text from the internet and is capable of generating high-quality natural language text.

Summarization is important because in today's fast-paced world, we are constantly bombarded with information. It can be difficult to keep up with everything that's going on, especially when there is so much to read. Summarization allows us to quickly and easily understand the key points of a long document, article, or report. It saves time and helps us make informed decisions about what to read and how to prioritize our time.

How ChatGPT Can Help with Summarization

ChatGPT can help with summarization because it is capable of generating high-quality natural language text. It can read a long document or article and summarize it in a way that is clear, concise, and easy to understand. This can save time and make it easier to process large amounts of information quickly. ChatGPT can also be customized to generate summaries that are tailored to specific industries or topics.

For example, if you work in the medical field, you might need to read through a lot of research papers and studies. ChatGPT can be trained on medical terminology and jargon to generate summaries that are specific to the medical field. This can save you a lot of time and make it easier to stay up-to-date on the latest research and developments in your field.

Another way that ChatGPT can help with summarization is by generating summaries of social media posts or news articles. With so much information being shared on social media and in the news, it can be overwhelming to keep up with everything. ChatGPT can read through these posts and articles and generate summaries that highlight the most important information.

Overall, ChatGPT is an incredibly powerful tool for summarization. It can help you save time, stay informed, and make better decisions about what to read and prioritize. As the technology continues to improve, we can expect to see even more applications for ChatGPT in the future.

Setting Up ChatGPT for Summarization

Before you can use ChatGPT for summarization, you will need to set it up on your computer. This involves creating an OpenAI account, installing necessary libraries, and configuring API keys.

Creating an OpenAI Account

The first step in setting up ChatGPT is to create an OpenAI account. OpenAI is an artificial intelligence research laboratory consisting of the for-profit corporation OpenAI LP and its parent company, the non-profit OpenAI Inc. OpenAI was founded in December 2015 by Tesla CEO Elon Musk, Greg Brockman, Ilya Sutskever, John Schulman, and Wojciech Zaremba. It aims to develop and promote friendly AI for the betterment of humanity.

To create an OpenAI account, you can visit the OpenAI website and follow the instructions for creating an account. Once you have created an account, you will be able to access the API and use ChatGPT for summarization.

Installing Necessary Libraries

After you have created your OpenAI account, you will need to install the necessary libraries on your computer. The libraries you need to install are Python and the OpenAI API library. Python is a high-level programming language that is widely used for web development, scientific computing, data analysis, artificial intelligence, and more. The OpenAI API library is a Python package that provides a convenient interface to the OpenAI API.

Installing Python is straightforward. You can download the latest version of Python from the official Python website and follow the installation instructions. To install the OpenAI API library, you can use pip, the package installer for Python. Open a terminal or command prompt and run the following command:

This will install the OpenAI API library and its dependencies.

Configuring API Keys

The final step in setting up ChatGPT is to configure your API keys. API keys are unique identifiers that are used to authenticate your requests to the OpenAI API. To generate an API key, you will need to log in to your OpenAI account and navigate to the API dashboard. From there, you can create a new API key and copy it to your clipboard.

It is important to store your API key securely on your computer and configure it to work with the OpenAI API library. One way to do this is to set the API key as an environment variable. You can do this by adding the following line to your .bashrc or .bash_profile file:

Replace 'your-api-key-here' with your actual API key. Once you have set the API key as an environment variable, you can use the OpenAI API library to generate summaries with ChatGPT.

Preparing Your Text for Summarization

Before you can generate a summary using ChatGPT, you will need to prepare your text. This involves selecting the right text, cleaning and formatting it, and breaking it down into smaller sections.

Preparing your text for summarization can be a time-consuming process, but it is essential if you want to get accurate and useful summaries. In this article, we will go over each step in detail and provide you with some tips and tricks to make the process easier.

Selecting the Right Text

The first step in preparing your text for summarization is to select the right text. You should choose text that is relevant to your needs and contains the information you want to summarize. This could be a long article, a report, or a series of documents.

When selecting your text, it's important to keep in mind the purpose of your summary. Are you summarizing the text for research purposes, or are you summarizing it for a general audience? The answer to this question will help you determine the level of detail you need to include in your summary.

Another thing to consider when selecting your text is the length. ChatGPT works best with shorter texts, so you may need to break down longer texts into smaller sections.

Cleaning and Formatting Your Text

Once you have selected your text, you will need to clean and format it. This involves removing any unnecessary formatting, such as tables or images, and making sure the text is in a readable format. You may also need to correct any spelling or grammar errors to ensure that ChatGPT can generate an accurate summary.

It's essential to clean and format your text carefully to avoid any errors in the summary. If your text contains too many errors, ChatGPT may not be able to generate a summary at all.

One way to make the cleaning and formatting process easier is to use text editing software that can automatically detect and correct errors.

Breaking Down the Text into Smaller Sections

Finally, you should break down your text into smaller sections. ChatGPT works best when it is given smaller chunks of text to summarize. You can break your text down into paragraphs or sections and submit them to ChatGPT one at a time.

Breaking down your text into smaller sections will also make it easier for you to review the summary and make any necessary revisions. It will also help you to identify any areas where the summary may be missing important information.

Overall, preparing your text for summarization is a crucial step in getting accurate and useful summaries. By selecting the right text, cleaning and formatting it carefully, and breaking it down into smaller sections, you can ensure that ChatGPT generates accurate and useful summaries that meet your needs.

Using ChatGPT to Generate Summaries

Once you have prepared your text, you are ready to use ChatGPT to generate your summary. This involves choosing the right model, adjusting parameters for summarization, and running the summarization process.

Choosing the Right Model

The first step in using ChatGPT to generate a summary is to choose the right model. ChatGPT has several models available, each with its own strengths and weaknesses. You should choose the model that best fits your needs and the type of text you are summarizing.

Adjusting Parameters for Summarization

The next step is to adjust the parameters for summarization. This involves specifying the length of the summary, the level of detail you want, and any other settings that may affect the output of the model.

Running the Summarization Process

Finally, you can run the summarization process. This involves submitting your text to the ChatGPT model and waiting for it to generate a summary. Once the summary is generated, you can review it to make sure it accurately captures the key points of the original text.

Using ChatGPT for summarization can be a powerful tool for managing information overload. By following the step-by-step guide in this article, you can set up ChatGPT on your computer, prepare your text for summarization, and generate high-quality summaries that are clear, concise, and easy to understand.

Take your idea to the next level with expert prompts.

Undetectable AI – Blog

AI Detector, Checker, and Humanizer Rated #1 by Forbes

Can ChatGPT Write Research Papers? Everything You Need to Know

Imagine being able to utilize AI to create research papers in a fraction of the time while maintaining the highest quality. Well, it’s possible and easier than ever. But can ChatGPT write research papers for you?

Using ChatGPT to write research papers was once considered unethical as it was seen as an attempt to manipulate the system. But things have changed. 

Today, there are numerous AI tools that we can use to humanize ChatGPT-generated content so that it sounds more natural and avoid algorithms designed to flag AI-generated texts.

Table of Contents

Can ChatGPT Write Research Papers?

The merging of AI with academic research opens a new chapter in the field of education. But can ChatGPT truly be used to write research papers?

The answer to this question lies in understanding the nuanced capabilities of this AI technology.

It’s not about replacing human intellect but rather supporting human authors, offering new perspectives when tackling complex tasks to simplify the writing process.

So at the end of the day? Yes. Theoretically, ChatGPT can write research papers. However, utilizing ChatGPT on its own without humanizing the text and revising it in a way that will have the human touch is not something you should consider doing. Let’s dig a little deeper.

What are the Benefits of Using ChatGPT to Write Research Papers?

There are many benefits of using ChatGPT to assist you when it comes to writing a research paper. It can’t be stressed enough that the key word here is “assist” when it comes to using ChatGPT to help you with your research paper.

But let’s discuss the benefits of having this kind of assistance.

Increased Efficiency

Time is a valuable commodity, especially in the area of academic research. ChatGPT has led to a breakthrough in time management and efficiency.

Imagine how much time you can save. ChatGPT’s ability to quickly generate and organize ideas, collect relevant information, and structure drafts makes the often tedious process of writing academic papers a more streamlined and efficient endeavor.

This not only saves time but also allows researchers and students to focus on a deeper analysis of their research results.

Improved Accuracy

Accuracy in research writing is paramount. Thanks to the advanced algorithms and access to a wide range of information from various databases, ChatGPT offers assistance in ensuring the accuracy of data interpretation and presentation.

By providing up-to-date references and cross-checking facts, ChatGPT acts as an additional layer of verification, reducing the likelihood of errors.

The ability to process and analyze large data sets can also help identify patterns and correlations that might be missed by the human eye, resulting in more accurate and reliable research outcomes.

Enhanced Creativity

One of the biggest benefits of using ChatGPT when writing academic papers is the enhancement of creativity. ChatGPT can stimulate creative thinking by offering a unique perspective and different approaches. It can propose new hypotheses and suggest methods that open up new possibilities for the author, pushing the boundaries of conventional academic writing.

This AI-driven creative boost is particularly beneficial in fields that thrive on innovative thinking and original approaches.

In summary, the integration of ChatGPT in research paper writing can provide significant benefits in terms of efficiency, accuracy, and creativity. Its ability to streamline the writing process, examine data, and offer creative approaches makes it an invaluable tool in the modern academic landscape.

How to Write Research Papers Using ChatGPT

1. define your research question or hypothesis.

The first step for your research paper is to define your research question or hypothesis. This fundamental step is crucial because it sets the direction of your entire research. 

ChatGPT can assist you in refining your hypothesis by providing insights on current trends, gaps in existing research, and suggesting various angles from which you can approach your topic.

This initial interaction with ChatGPT sets the tone for a focused and purpose-driven research paper.

2. Conduct Literature Review

Once you have established your hypothesis, you need to conduct a comprehensive literature search. ChatGPT can help with this task. Its large data sets help you find relevant literature, compile research results, and identify areas that may need further research. This AI-driven approach ensures a thorough review of existing knowledge, laying a robust foundation for your research.

3. Gather and Analyze Data

Data gathering and analysis are pivotal in any research paper. ChatGPT can streamline this step by suggesting appropriate data collection methods and tools based on your hypothesis. Furthermore, it can help with initial analysis of data and identify potential patterns or anomalies. This function is particularly helpful when processing and interpreting large data sets because it makes data analysis more manageable and efficient.

4. Organize Your Paper

Now you need to organize all the information in a meaningful way so that your research paper is coherent and logical. ChatGPT can aid in structuring content, suggest relevant headings and subheadings, and ensure that your arguments flow logically from one section to the next. This AI-guided structuring helps to maintain clarity and focus, making your paper more comprehensible and impactful.

5. Draft Your Paper

Drafting the paper is where ChatGPT’s capabilities truly shine. You can use the AI program to create drafts based on the data collected while ensuring that your writing maintains a consistent tone and style. ChatGPT’s ability to generate ideas and draft sections of the paper can significantly expedite the writing process, allowing you to focus on the critical aspects of your research.

6. Edit and Proofread Your Paper

The final step involves editing and proofreading your academic paper, which is crucial for ensuring the quality and readability of your work. ChatGPT can assist in this process by finding grammatical errors, suggesting improvements in sentence structure, and even checking the consistency of terminology and formatting. This ensures that your research paper is not only informative but also presented professionally.

In conclusion, writing an academic research paper with ChatGPT requires a structured and systematic approach, from defining your research question to the final stages of editing and proofreading. ChatGPT’s AI-driven features enhance each phase of the process, making research writing more efficient, comprehensive, and high-quality.

Using ChatGPT Effectively to Write Research Papers

1. provide chatgpt with clear instructions.

To utilize ChatGPT’s full potential when writing research papers, you need to provide the program with clear and concise instructions. You have to formulate your requirements and expectations in such a way that the AI can understand them and react accordingly.

Whether you’re seeking assistance in brainstorming, drafting, or editing, if you provide accurate information, ChatGPT can generate more targeted responses.

Provide details about the topic, the scope of the content, the style of writing, or any particular points that need to be addressed. Clear instructions provide guidance and enable ChatGPT to align its responses with your academic objectives.

2. Review ChatGPT’s Output Carefully

While ChatGPT can produce impressive content, you must always check the relevance and accuracy of the information provided. Also, ensure that the content meets current academic standards and aligns with your research goals.

Additionally, you can use ChatGPT to identify areas that may need refinement or further elaboration.

It’s not about simply accepting AI-generated content, but rather about carefully reviewing the results and then integrating it thoughtfully into your research while maintaining your academic integrity.

3. Be Prepared to Edit and Revise ChatGPT’s Output

You always need to revise your work, especially when it involves AI-generated content. While ChatGPT is a powerful tool, it is not infallible.

AI content may need to be adjusted to better fit the tone, style, or specific requirements of the task. Editing and revising could range from making minor tweaks to undertaking significant rewrites. 

You must customize ChatGPT results so that they meet academic standards. This step is crucial to ensure that the work reflects your research findings, your personal academic style, and your unique perspective.

Using ChatGPT when writing academic papers requires a mix of clear communication, critical thinking, and thoughtful revision. By providing clear instructions, carefully reviewing the AI’s output, and being prepared to make necessary revisions, you will be able to leverage ChatGPT’s features to improve your research writing process while maintaining the essence of your academic work.

This approach ensures that the integration of AI in your research is both effective and harmonious.

Writing Your Thesis with ChatGPT: Is It Feasible?

Writing a thesis with ChatGPT raises an interesting question: Can the AI program handle this task with the care and accuracy required?

In summary, it is possible if you follow certain rules and know the strengths and weaknesses of ChatGPT. You have to assess your work meticulously, formulate a good thesis, and always revise your work afterward.

As the field of academic research continues to evolve, ChatGPT will soon revolutionize the education sector. Artificial intelligence heralds a new era in which AI and human intellect collaborate to achieve academic excellence.

If you plan to use ChatGPT to help assist you in the production of your research paper, using the Undetectable AI Humanizer tool as part of that process is highly recommended. You need to ensure that whatever assistance you have from ChatGPT while writing your research paper doesn’t go to waste by having your paper flagged by AI detectors once your assignment is turned in.

Use the Undetectable AI Humanizer to make sure that isn’t a possibility.

1. How does ChatGPT contribute to the accuracy and reliability of research papers?

ChatGPT contributes to the accuracy and reliability of research by providing data-driven insights, timely information, and facts from a variety of sources. It can assist in fact-checking, suggesting relevant literature, and identifying inconsistencies in arguments or data. However, it should be noted that while ChatGPT is a valuable tool for enhancing accuracy, the ultimate responsibility for verifying facts and data rests with the researcher.

2. In what ways can ChatGPT manage complex and diverse research topics?

ChatGPT can manage complex and diverse research topics by providing a comprehensive overview of a wide range of topics. It can generate ideas, suggest research priorities, and provide insights on various topics. Because ChatGPT can quickly process and analyze large amounts of information, it is well suited to complex research questions as it offers new perspectives and helps researchers explore new areas or interdisciplinary approaches.

3. Are there any limitations to be aware of when using ChatGPT for academic writing?

Yes, there are of course limitations to using ChatGPT for academic writing. ChatGPT may not always have access to the latest research developments or specific subject matter expertise. Its understanding of context or subtle nuances is also limited. AI may not be able to provide detailed in-depth analysis comparable to a human expert. Additionally, the work could contain critical errors if you rely too heavily on ChatGPT. Researchers should use ChatGPT as a supplementary tool, combining its outputs with their own expertise and critical thinking.

4. How can researchers ensure the originality of work when incorporating ChatGPT’s assistance?

To ensure originality when using ChatGPT, researchers should use the AI’s output as a starting point or a source of inspiration. Content cannot simply be adopted. AI-generated content needs a unique approach and a fresh, human perspective. Additionally, it’s crucial to properly cite any sources provided by ChatGPT and avoid copying text verbatim without modification. It’s best to run a plagiarism check to help maintain the uniqueness of the work.

5. Is ChatGPT versatile enough to support research across various academic disciplines?

ChatGPT is versatile and can support researchers in a wide range of academic disciplines. AI can provide information in various fields, from humanities to the sciences. However, the depth of its knowledge and the specificity of the information it provides can vary depending on the discipline. For highly specialized or emerging areas, additional expertise may be needed alongside ChatGPT’s assistance.

6. Can ChatGPT play a role in the analysis and interpretation of data in research papers?

ChatGPT can assist in analyzing and interpreting data in research papers. It can assist in identifying patterns, suggesting analytical approaches, and providing preliminary interpretations of data. However, it should be noted that ChatGPT is not a replacement for specialized software or the expertise of a data analyst. Researchers should use ChatGPT as a complementary tool, complementing it with their own analytical skills and tools specific to their field of research.

• What is ChatGPT?
• How to Use Google Gemini

How to use ChatGPT to summarize a book or article

ChatGPT, OpenAI's artificial intelligence chatbot, seems to be able to do lots of things. You can have it explain complex concepts in simple terms, ask it for a cocktail recipe, or even have it write up Excel formulas for your spreadsheets.

Can ChatGPT summarize books and articles?

How to use chatgpt to summarize a book, how to use chatgpt to summarize an article.

But what about books and articles? Can ChatGPT help you make sense of a news article on a niche topic? Or summarize the plot of a given novel? Yes, but it does have limitations you need to know about, and there are a few things to keep in mind before using this AI chatbot for this purpose.

In this guide, we'll go over the caveats of using ChatGPT for summarizing articles and books. And we'll also show you exactly how to get ChatGPT to summarize them for you, in a few quick steps.

What You Need

An OpenAI account

A web browser

A link to an article, or a book's title and its author's name

Yes, ChatGPT can summarize books and articles, but there are notable limitations to its ability to do so.

For one thing, it can generate summaries or other responses that contain factual errors. In fact, ChatGPT admits as much on its own interface: The AI chatbot's New Chat screen features the phrase "May occasionally generate incorrect information" listed as one of its limitations. So you really do need to be careful about treating ChatGPT's responses as absolute truth. And you will need to do some fact-checking of your own to confirm that its responses are indeed correct.

Another issue? ChatGPT also has an oddly specific limit on its knowledge: It has what it calls a "knowledge cutoff" in which it knows very little about the world after September 2021. So if you ask for a summary of a book that was published after September 2021, it's likely that ChatGPT will be unable to provide that summary and instead offer up a limited response like the one in the screenshot below.

The screenshot actually shows both of the limitations described above in action. In the conversation with ChatGPT below, we asked ChatGPT to provide a summary of Prince Harry's book Spare , which was published on January 10, 2023. In response, ChatGPT instead referred to its knowledge cutoff and said that the book does not exist. It then went on to recommend another book by Prince Harry, which actually does not exist, at least not as a book. ChatGPT tried to recommend The Me You Can't See: A Memoir as a book that was published in 2021, but to our knowledge, such a book doesn't exist, though there is a documentary series with a similar name that involves Prince Harry and Oprah Winfrey that was released in 2021.

Using ChatGPT to summarize a book is actually pretty simple. You'll usually just need to include the title of the book and the name of its author in your request for ChatGPT to summarize it.

Step 1: Choose a book you want summarized by ChatGPT. Be sure to choose a book that was published before September 2021 and confirm that you have the correct title and name of the book's author.

Having this information and picking a book published before September 2021 will help ensure that ChatGPT gives you the most accurate response and doesn't send an error message.

Step 2: Open up your web browser on your PC and go to the ChatGPT website . Then log into your OpenAI account, if you haven't already.

You should now see a New chat screen on ChatGPT's web interface.

• How to build your own GPT chatbot without coding
• How to cancel your ChatGPT Plus subscription
• How to search ChatGPT conversations

Step 3: On the New chat screen, select the Send a message... text box at the bottom of your screen and begin typing your request for a summary of your desired book into that text box.

In our example, we submitted the following request to ChatGPT: Please summarize the book "Great Expectations" by Charles Dickens.

Once you're done writing your request, press the Enter key on your keyboard.

Step 4: Once you've submitted your request, ChatGPT should immediately begin writing its response. In seconds, you should then be presented with ChatGPT's summary of the book you asked about.

Please remember that ChatGPT's responses aren't always correct, that they can sometimes contain factual errors, and that we strongly recommend that you still read the book to confirm whether or not this AI chatbot actually produced an accurate summary of the book.

Using ChatGPT to summarize a news article is pretty easy, and it seems to be even easier than asking the AI chatbot for a summary of a book. That's especially true if you have a link to an online news article. In the steps below, we'll show you how to do just that.

Step 1: Open up your web browser on your PC and navigate to the article you want summarized. Once the article appears, copy its URL (the article's link) from the address bar at the top of your browser (to do so, highlight the URL and then press CTRL + C on your keyboard).

Step 2: Once the URL is copied to your clipboard, open another tab in your browser and go to the ChatGPT website . Then log into your OpenAI account.

You should then be presented with a New chat screen on ChatGPT's website.

Step 3: On the New chat screen, select the Send a message... text box at the bottom of your screen and type in your message to ChatGPT requesting a summary of your chosen article. In your message, be sure to include the article URL you copied earlier (to do so, you'll need to paste the URL into the text box by selecting the text box and then pressing CTRL + V on your keyboard).

In our example, we submitted the following message to ChatGPT: Please summarize this article: https://www.digitaltrends.com/computing/chatgpt-is-getting-internet-access-with-plugins/

Once you're done composing your message to ChatGPT, press the Enter key on your keyboard to send it.

Step 4: Once you've sent your request to ChatGPT, it should only take a few seconds for it to respond with its summary of the article you chose.

Don't forget: While ChatGPT can usually provide you with a summary of an article, that doesn't automatically mean the summaries it produces are always fully accurate. ChatGPT can and sometimes does make mistakes. As a result, we still firmly recommend that you read the article or articles in question, so that you can see for yourself whether or not ChatGPT really did give you an accurate summary.

Editors' Recommendations

• 8 AI chatbots you should use instead of ChatGPT
• How to delete your ChatGPT account
• How to get GPT-4 for free
• The best ChatGPT plug-ins you can use
• Apple finally has a way to defeat ChatGPT
• Artificial Intelligence
• How-To Guides

The ChatGPT chatbot is an innovative AI tool developed by OpenAI. As it stands, there are two main versions of the software: GPT-4 and GPT-3.5. Toe to toe in more ways than one, there are a couple of key differences between both versions that may be deal-breakers for certain users. But what exactly are these differences? We’re here to help you find out. 

We’ve put together this side-by-side comparison of both ChatGPT versions, so when you’re done reading, you’ll know what version makes the most sense for you and yours. What are GPT 3.5 and GPT-4?

ChatGPT, the AI-powered chatbot that went viral at the start of last year and kicked off a wave of interest in generative AI tools, no longer requires an account to use.

Its creator, OpenAI, launched a webpage on Monday that lets you begin a conversation with the chatbot without having to sign up or log in first.

In recent years, the listening time required by a piece of AI to clone someone’s voice has been getting shorter and shorter.

It used to be minutes, now it’s just seconds.

Secure Your Spot in Our PCA Online Course Starting on April 02 (Click for More Info)

Research Paper Summary Using ChatGPT (Example)

Hi! In this tutorial, I will show you how to use ChatGPT to summarize a research paper .

Since ChatGPT cannot access an external URL, we cannot give it the link to a research paper along with a summary instruction prompt.

Therefore, we will need to make use of ChatGPT Summary for Chrome extension . The installation and usage of the extension are straightforward, but if you need assistance, then take a look at this tutorial article .

Here is an overview:

Let’s jump into the discussion!

Get Research Paper URL

In order to be able to use ChatGPT Summary for Chrome extension to summarize a research paper, the paper has to be hosted online and be in the form of a web article because, from my experience, it cannot summarize a hosted PDF document .

However, if you would like to know how to use ChatGPT to summarize a PDF document, then you can read this tutorial article .

The research paper we will summarize in this tutorial is from ScienceDirect . It is a paper about artificial intelligence and unemployment in high-tech developed countries. You can access the paper here .

  Now that we have accessed the research paper, we can ask ChatGPT to summarize it for us. First, though, we will need to make a slight modification to the prompt.  

Modify Summary Prompt

The default prompt that comes with the ChatGPT Summary for Chrome extension instructs ChatGPT to summarize an article into 5 main points.

In our case, we want it to summarize the research paper as one short paragraph. Therefore, we will adjust the prompt setting to reflect that.

So, go to the extension icon on your Google Chrome browser menu bar, right-click on it, and select “Options”. In the prompt area, enter the following prompt: Summarize the main points of the paper as one short paragraph and click “Save Prompt”.

  With that, ChatGPT should summarize the research paper in a short paragraph as instructed.  

Summarize Research Paper

Now that we have adjusted our prompt instruction, it is time to run the extension on the research paper.

Simply go back to the research paper and click the ChatGPT Summary for Chrome extension icon, and it will automatically summarize the paper in a popup widget like so.

  It certainly did a good job summarizing the entire paper in one paragraph. You can experiment with the prompt and see in what other ways the research paper can be summarized using ChatGPT.  

Video, Further Resources & Summary

Do you need more explanations on how to summarize a research paper using ChatGPT? Then you should have a look at the following YouTube video of the Statistics Globe YouTube channel.

In the video, we explain in some more detail how to summarize a research paper using ChatGPT.

The YouTube video will be added soon.

Furthermore, you could have a look at some of the other interesting ChatGPT-based tutorials on Statistics Globe:

• Best ChatGPT Prompts for Writing a Cover Letter (5 Examples)
• Best ChatGPT Prompts (7 Examples)
• How to Use ChatGPT for Story Writing (Example)
• How to Write Better ChatGPT Prompts (Examples)
• How to Use ChatGPT for Coding (Examples)
• Introduction to Python Programming

This post has shown how to summarize a research paper using ChatGPT . I hope you found this post helpful! In case you have further questions, you may leave a comment below.

This page was created in collaboration with Ifeanyi Idiaye. You might check out Ifeanyi’s personal author page to read more about his academic background and the other articles he has written for the Statistics Globe website.

Subscribe to the Statistics Globe Newsletter

Get regular updates on the latest tutorials, offers & news at Statistics Globe. I hate spam & you may opt out anytime: Privacy Policy .

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Post Comment

I’m Joachim Schork. On this website, I provide statistics tutorials as well as code in Python and R programming.

Statistics Globe Newsletter

Get regular updates on the latest tutorials, offers & news at Statistics Globe. I hate spam & you may opt out anytime: Privacy Policy .

Related Tutorials

Scatterplot of PCA in R (2 Examples)

Draw Biplot of PCA in Python (Example)

• 5 Best AI Research Paper Summarizers (May 2024)

Unite.AI is committed to rigorous editorial standards. We may receive compensation when you click on links to products we review. Please view our affiliate disclosure .

Table Of Contents

In the fast-paced world of academic research, keeping up with the ever-growing body of literature can be a daunting task. Researchers and students often find themselves inundated with lengthy research papers, making it challenging to quickly grasp the core ideas and insights. AI-powered research paper summarizers have emerged as powerful tools, leveraging advanced algorithms to condense lengthy documents into concise and readable summaries.

In this article, we will explore the top AI research paper summarizers, each designed to streamline the process of understanding and synthesizing academic literature:

1. Tenorshare AI PDF Tool

Tenorshare AI PDF Tool is a cutting-edge solution that harnesses the power of artificial intelligence to simplify the process of summarizing research papers. With its user-friendly interface and advanced AI algorithms, this tool quickly analyzes and condenses lengthy papers into concise, readable summaries, allowing researchers to grasp the core ideas without having to read the entire document.

One of the standout features of Tenorshare AI PDF Tool is its interactive chat interface, powered by ChatGPT. This innovative functionality enables users to ask questions and retrieve specific information from the PDF document, making it easier to navigate and understand complex research papers. The tool also efficiently extracts critical sections and information, such as the abstract, methodology, results, and conclusions, streamlining the reading process and helping users focus on the most relevant parts of the document.

Key features of Tenorshare AI PDF Tool:

• AI-driven summarization that quickly condenses lengthy research papers
• Interactive chat interface powered by ChatGPT for retrieving specific information
• Automatic extraction of critical sections and information from the paper
• Batch processing capabilities for handling multiple PDF files simultaneously
• Secure and private, with SSL encryption and the option to delete uploaded files

Elicit is an AI-powered research assistant that improves the way users find and summarize academic papers. With its intelligent search capabilities and advanced natural language processing, Elicit helps researchers quickly identify the most relevant papers and understand their core ideas through automatically generated summaries.

By simply entering keywords, phrases, or questions, users can leverage Elicit's AI algorithms to search through its extensive database and retrieve the most pertinent papers. The tool offers various filters and sorting options, such as publication date, study types, and citation count, enabling users to refine their search results and find exactly what they need. One of Elicit's most impressive features is its ability to generate concise summaries of the top papers related to the search query, capturing the key findings and conclusions and saving researchers valuable time.

Key features of Elicit:

• Intelligent search that understands the context and meaning of search queries
• Filters and sorting options for refining search results
• Automatic summarization of the top papers related to the search query
• Detailed paper insights, including tested outcomes, participant information, and trustworthiness assessment
• Inline referencing for transparency and accuracy verification

3. QuillBot

QuillBot is an AI-powered writing platform that offers a comprehensive suite of tools to enhance and streamline the writing process, including a powerful Summarizer tool that is particularly useful for condensing research papers. By leveraging advanced natural language processing and machine learning algorithms, QuillBot's Summarizer quickly analyzes lengthy articles, research papers, or documents and generates concise summaries that capture the core ideas and key points.

One of the key advantages of QuillBot's Summarizer is its ability to perform extractive summarization, which involves identifying and extracting the most critical sentences and information from the research paper while maintaining the original context. Users can customize the summary length to be either short (key sentences) or long (paragraph format) based on their needs, and the output can be generated in either a bullet point list format or as a coherent paragraph. This flexibility allows researchers to tailor the summary to their specific requirements and preferences.

Key features of QuillBot's Summarizer:

• AI-powered extractive summarization that identifies and extracts key information
• Customizable summary length (short or long) to suit different needs
• Bullet point or paragraph output for flexible formatting
• Improved reading comprehension by condensing the paper into its core concepts
• Integration with other QuillBot tools, such as Paraphraser and Grammar Checker, for further enhancement

4. Semantic Scholar

Semantic Scholar is a free, AI-powered research tool developed by the Allen Institute for AI that improves the way researchers search for and discover scientific literature. By employing advanced natural language processing, machine learning, and machine vision techniques, Semantic Scholar provides a smarter and more efficient way to navigate the vast landscape of academic publications.

One of the standout features of Semantic Scholar is its ability to generate concise, one-sentence summaries of research papers, capturing the essence of the content and allowing researchers to quickly grasp the main ideas without reading lengthy abstracts. This feature is particularly useful when browsing on mobile devices or when time is limited. Additionally, Semantic Scholar highlights the most important and influential citations within a paper, helping researchers focus on the most relevant information and understand the impact of the research.

Key features of Semantic Scholar:

• Concise one-sentence summaries of research papers for quick comprehension
• Identification of the most influential citations within a paper
• Personalized paper recommendations through the “Research Feed” feature
• Semantic Reader for in-line citation cards with summaries and “skimming highlights”
• Personal library management with the ability to save and organize papers

5. IBM Watson Discovery

IBM Watson Discovery is a powerful AI-driven tool designed to analyze and summarize large volumes of unstructured data, including research papers, articles, and scientific publications. By harnessing the power of cognitive computing, natural language processing, and machine learning, Watson Discovery enables researchers to quickly find relevant information and gain valuable insights from complex documents.

One of the key strengths of IBM Watson Discovery is its ability to understand the context, concepts, and relationships within the text, allowing it to identify patterns, trends, and connections that may be overlooked by human readers. This makes it easier to navigate and summarize complex research papers, as the tool can highlight important entities, relationships, and topics within the document. Users can create customizable queries, filter, and categorize data to generate summaries of the most relevant research findings, and the tool's advanced search capabilities enable precise searches and retrieval of specific information from large document libraries.

Key features of IBM Watson Discovery:

• Cognitive capabilities that understand context, concepts, and relationships within the text
• Customizable queries and filtering for generating summaries of relevant research findings
• Relationship identification to highlight important entities, relationships, and topics
• Significant time-saving by automating the discovery of information and insight

Empowering Researchers with AI-Driven Summarization Tools

The emergence of AI-powered research summarizers has transformed the way researchers and academics approach scientific literature. By leveraging advanced natural language processing, machine learning, and cognitive computing, these innovative tools enable users to quickly find, understand, and summarize complex research papers, saving valuable time and effort.

Each of these AI research summarizers offers unique features and benefits that cater to researchers' diverse needs. As these tools continue to evolve and improve, they will undoubtedly play an increasingly crucial role in empowering researchers to navigate the ever-expanding universe of scientific knowledge more efficiently and effectively.

5 Best AI SOP (Standard Operating Procedures) Generators in 2024

Alex McFarland is an AI journalist and writer exploring the latest developments in artificial intelligence. He has collaborated with numerous AI startups and publications worldwide.

You may like

5 Best B2B Customer Support Tools (May 2024)

5 Best AI Apps for Couples (May 2024)

10 Best AI Shopify Tools (May 2024)

10 Best AI Business Plan Generators (May 2024)

10 Best AI Tools for Google Sheets (May 2024)

Recent Posts

• Itamar Friedman, CEO & Co-Founder of CodiumAI – Interview Series
• Hungry for Data: How Supply Chain AI Can Reach its Inflection Point
• Hostinger Review – Is This The Most Affordable Premium Webhost?
• What is AlphaFold 3? The AI Model Poised to Transform Biology

Featured Topics

Featured series.

A series of random questions answered by Harvard experts.

Explore the Gazette

Read the latest.

Roger Ware Brockett, 84

Providing community support

A change of mind, heart, and soul

“Telling people not to use ChatGPT is not preparing people for the world of the future,” said Sam Altman, CEO of OpenAI.

Niles Singer/Harvard Staff Photographer

Did student or ChatGPT write that paper? Does it matter?

Sam Altman, CEO of firm that developed app, says ethics do matter, but they need to be rethought (and AI isn’t going away)

Harvard Correspondent

Colleges and universities have been wrestling with concerns over plagiarism and other ethical questions surrounding the use of AI since the emergence of ChatGPT in late 2022.

But Sam Altman, whose company, OpenAI, launched the chatbot app, said during a campus visit Wednesday that AI is such a powerful tool that higher education would be doing its students a disservice by turning its back on it — if that were even possible now. And some of the old rules of ethics will need to be rethought.

“Cheating on homework is obviously bad,” said Altman. “But what we mean by cheating and what the expected rules are does change over time.”

Altman discussed AI in the academy, along with the subtleties of using ChatGPT and other generative AI tools, while at the University to receive the Experiment Cup from Xfund , an early stage venture capital firm. That event was sponsored by the John A. Paulson School for Engineering and Applied Science, Harvard Business School, and the Institute for Business in Global Society ( BiGS ). It featured a conversation between Altman and Xfund co-founder Patrick Chung ’96.

Speaking to the Gazette before the Cup presentation, Altman likened the initial uproar at schools over ChatGPT to the ones that arose after the arrival of calculators and, later, search engines like Google. “People said, ‘We’ve got to ban these because people will just cheat on their homework,’” he said.

Altman, who left Stanford at 19 to start Loopt, a location-sharing social media app, said the reaction to calculators, for instance, was overblown. “If people don’t need to calculate a sine function by hand again … then mathematical education is over,” he said, with a gentle half-smile on his face.

Altman helped launch OpenAI in 2015 and its wildly influential ChatGPT — which can write papers and generate computer programs, among other things — before being removed in 2023 and then reinstated four days later as the company’s CEO.

ChatGPT, he said, has the potential to exponentially increase productivity in the same way calculators freed users from performing calculations by hand, calling the app “a calculator for words.”

He warned, “Telling people not to use ChatGPT is not preparing people for the world of the future.”

Following a bit of back-and-forth about how the ethics of using ChatGPT and other generative AI may differ in various disciplines, Altman came down hard in favor of utility, praising AI’s massive potential in every field.

“Standards are just going to have to evolve,” he said. He dismissed the notion that ChatGPT could be used for writing in the sciences, where the emphasis is on the findings, but not in the humanities, where the expression of ideas is central.

“Writing a paper the old-fashioned way is not going to be the thing,” he said. “Using the tool to best discover and express, to communicate ideas, I think that’s where things are going to go in the future.”

Altman, who last month joined the Department of Homeland Security’s Artificial Intelligence Safety and Security Board , said ethics remains a concern, and one that has yet to be resolved.

“There will be a conversation about what are the absolute limits of the tool, how do we as a society … negotiate ‘Here is what AI systems can never do.’ Where do we set the defaults? How much does an individual user get to move things around within those boundaries? How do we think about different countries’ laws?”

However, that discussion should not slow the development of AI. Instead, Altman described parallel tracks.

“Generally speaking, I do think these are tools that should do what their users want,” he said, before adding an important, if less than specific, caveat: “But there are going to have to be real limits.”

Share this article

You might like.

Memorial Minute — Faculty of Arts and Sciences

Harvard Allston Partnership Fund awards grants to 26 Allston-Brighton nonprofits

Choosing Harvard took LyLena Estabine down an uncertain path. The former student co-president has no regrets.

Epic science inside a cubic millimeter of brain

Researchers publish largest-ever dataset of neural connections

Excited about new diet drug? This procedure seems better choice.

Study finds minimally invasive treatment more cost-effective over time, brings greater weight loss

How far has COVID set back students?

An economist, a policy expert, and a teacher explain why learning losses are worse than many parents realize

Have a thesis expert improve your writing

Check your thesis for plagiarism in 10 minutes, generate your apa citations for free.

• Knowledge Base

Using AI tools

• How to Write a Paper with ChatGPT | Tips & Examples

How to Write a Paper with ChatGPT | Tips & Examples

Published on 17 August 2023 by Koen Driessen .

A good research paper demonstrates your academic writing skills and knowledge of your chosen topic.

Your research paper should be based on in-depth independent research. However, generative AI tools like ChatGPT can be effectively used throughout the research process to:

• Brainstorm research questions
• Develop a methodology
• Create an outline
• Find sources
• Summarise and paraphrase text
• Provide feedback

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Brainstorming research questions, developing a methodology, creating an outline, finding sources, summarising or paraphrasing text, receiving feedback, other interesting articles, frequently asked questions.

You can use ChatGPT to help formulate research questions , brainstorm potential thesis statements , or narrow down a broad subject area to a specific topic that interests you.

However, make sure that the outputs make sense and meet the requirements of your assignment. You can adapt these outputs manually, or use further prompts to ensure they meet your needs. For more information, see our tips for creating effective ChatGPT prompts .

• Generate three possible research questions for a research paper on the following topic: “The role of virtual reality in enhancing training and skill development in various industries.”
• “What are the cognitive and skill development benefits of virtual reality-based training compared to traditional training methods across different industries?”
• “How does the level of immersion in virtual reality training impact knowledge retention and skill transfer in fields such as medicine, aviation, and manufacturing?”
• “What are the key factors that influence the successful integration and adoption of virtual reality technology for training purposes in diverse industries, and what challenges hinder its widespread implementation?”

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Your paper should clearly indicate the data collection and analysis methods you used in your research.

You can use ChatGPT to help decide what kind of research you will conduct (e.g., qualitative or quantitative ), how you will analyse the data, and how you will mitigate potential research bias .

Bear in mind that you should be able to rationalize your chosen methodology . For example, you should ensure that it is either a standard methodology in the field or can be justified.

• Suggest three quantitative research methods appropriate to the following question: “What are the cognitive and skill development benefits of virtual reality-based training compared to traditional training methods in the field of healthcare?”
• Experimental Study: Conduct a randomized controlled trial (RCT) where healthcare professionals are randomly assigned to either a virtual reality-based training group or a traditional training group. Both groups would receive training in specific healthcare tasks, and their cognitive and skill development outcomes would be measured and compared. Performance metrics, such as accuracy, speed, and efficiency, could be quantitatively assessed to determine the benefits of virtual reality-based training over traditional methods.
• Survey Research: Design a comprehensive survey targeting healthcare professionals who have undergone both virtual reality-based training and traditional training. The survey could include Likert-scale questions and structured response options to gauge their perceived cognitive and skill development benefits from each training method. Additionally, open-ended questions could be included to gather qualitative insights on specific strengths and weaknesses of each training approach.
• Task Performance Analysis: Implement a task-based assessment that simulates real-life healthcare scenarios, where participants are evaluated on their ability to handle critical situations. Participants would go through both virtual reality-based training and traditional training before being assessed in the simulated scenarios. Their performance in the assessment could be objectively measured using predefined criteria, allowing for a quantitative comparison of skill development between the two training methods.

A clear outline can help structure your paper, organise your arguments effectively, and make your research more efficient.

You can use generative AI tools like ChatGPT to formulate a potential outline to help guide your research. You can be as specific as you like, giving an overview of your research objectives , main arguments, and any evidence you intend to include in your paper.

However, ChatGPT outputs sometimes contain factual inaccuracies. It’s important to check the headings and subheadings suggested in the output and adapt them to meet your needs.

• Create an outline for an academic research paper with the following research question:“What are the cognitive and skill development benefits of virtual reality-based training compared to traditional training methods in the field of healthcare?” The paper will be about 10 pages long.

B. Existing studies on cognitive and skill development benefits of virtual reality training

C. Comparison of virtual reality-based training with traditional training methods

D. Gaps in the literature and research limitations

III. Theoretical Framework

A. Theoretical basis for cognitive and skill development in training

B. Application of cognitive theories to virtual reality-based training

C. The role of immersive learning environments in skill acquisition

D. Conceptual framework for the study

IV. Research Methodology

A. Research design and approach (e.g., quasi-experimental, pre-post intervention)

B. Selection of participants and sample size calculation

C. Description of the virtual reality-based training and traditional training programs

D. Instruments and measures for assessing cognitive and skill development

E. Data collection procedures and ethical considerations

V. Data Analysis

A. Overview of statistical methods used for data analysis

B. Treatment of data and data cleaning

C. Quantitative analysis of cognitive and skill development outcomes

D. Presentation of findings with charts and tables

VI. Results

A. Presentation of the results of the cognitive and skill development assessments

B. Comparison of outcomes between virtual reality-based training and traditional training

C. Statistical significance and effect size analysis

D. Interpretation of the results and implications

VII. Discussion

A. Interpretation of findings in the context of the research question

B. Comparison of results with existing literature

C. Explanation of how virtual reality-based training impacts cognitive processes and skill development

D. Limitations of the study and areas for future research

VIII. Practical Implications

A. Practical applications of virtual reality-based training in healthcare settings

B. Recommendations for integrating virtual reality training in healthcare education

C. Benefits of virtual reality for continuous professional development in healthcare

IX. Conclusion

A. Summary of key findings

B. Reiteration of the research question and hypothesis

C. Contributions of the study to the field of healthcare training

D. Concluding remarks

X. References

You can use ChatGPT to help find sources relevant to your research. However, it’s important to be aware of the limitations of ChatGPT (for example, it occasionally cites nonexistent sources).

Instead of asking ChatGPT for specific source recommendations, try asking it for recommendations on the types of sources relevant to your research topic. You can also use it to generate a list of relevant keywords to use in your literature search.

• What types of sources could I use to write a research paper on the following research question? “What are the cognitive and skill development benefits of virtual reality-based training compared to traditional training methods in the field of healthcare?”
• Academic Journals: Peer-reviewed journals are valuable sources of scholarly research. Look for articles in medical education, cognitive psychology, virtual reality applications in healthcare, and technology-enhanced learning.
• Research Studies: Seek out published research studies that specifically investigate the impact of virtual reality-based training on cognitive and skill development in healthcare professionals.
• Systematic Reviews and Meta-Analyses: These studies provide comprehensive overviews of existing research on a specific topic, making them a valuable source for summarizing and analyzing the overall evidence.
• Books and Book Chapters: Academic books and book chapters authored by experts in the fields of medical education, virtual reality, cognitive science, or instructional design can provide in-depth insights and theoretical foundations.
• Conference Proceedings: Proceedings from relevant conferences and symposiums may present the latest findings and advancements in virtual reality-based training and healthcare education.
• Educational Institution Websites: Universities and medical schools often publish research papers, case studies, and whitepapers related to medical education and virtual reality training.
• Industry Reports: Reports and whitepapers from healthcare organizations or technology companies exploring the benefits of virtual reality-based training in the healthcare sector.
• Government and Non-profit Organisations: Look for reports and publications from government agencies and non-profit organisations that focus on medical education and healthcare training.
• Online Databases: Utilize academic databases such as PubMed, Google Scholar, IEEE Xplore, and ScienceDirect to search for peer-reviewed research articles and conference papers.
• Expert Interviews: If possible, conduct interviews with experts in medical education, cognitive science, and virtual reality to gain valuable insights and perspectives.
• Surveys and Polls: Refer to reputable surveys and polls related to virtual reality-based training in the healthcare industry for additional statistical data.

You can use ChatGPT to paraphrase or summarise text. This can help you to condense sources to their most important points and explore new ways of expressing your ideas.

Alternatively you can use the more specialised tools featured on Scribbr’s AI writing resources page, which are designed specifically for these purposes and will give a smoother user experience.

• Provide a short summary of the following text:“One of the most profound applications of VR in healthcare is its use in surgical training. Traditionally, surgical training has relied on a combination of didactic lectures, observation of procedures, and hands-on experience in the operating room. However, these methods may not provide the ideal environment for novices to develop the necessary dexterity and spatial understanding essential for complex surgeries. VR overcomes these limitations by offering immersive, true-to-life simulations that allow surgeons to navigate through intricate anatomical structures, practice incisions, and learn to manage unexpected challenges that may arise during surgeries. This hands-on experience in a risk-free environment not only enhances technical skills but also builds confidence and reduces the learning curve for new surgical techniques.”
• Virtual Reality (VR) has a profound impact on healthcare, particularly in surgical training, as it provides immersive simulations that help surgeons develop dexterity, spatial understanding, and confidence for complex surgeries in a risk-free environment, ultimately enhancing their technical skills and reducing the learning curve.

When you’ve finished writing your research paper, you can use ChatGPT to receive feedback. You can be as specific as you like, selecting particular aspects the output should focus on (e.g., tone, clarity of structure, appropriateness of evidence to support your arguments).

You can also use ChatGPT to check grammar, spelling, and punctuation. However, it’s not designed for this purpose and occasionally misses errors. We recommend using a more specialised tool like Scribbr’s free grammar checker . Or, for more comprehensive feedback, Scribbr’s proofreading and editing service .

• Feedback the above text with regard to appropriateness of evidence and coherence of structure.
• The research paper demonstrates a well-structured and coherent presentation of the cognitive and skill development benefits of VR-based surgical training. The introduction effectively sets the context and rationale, while the literature review incorporates relevant historical and empirical evidence. The theoretical framework strengthens the foundation for the analysis.The methodology section could provide more details on the research design and study selection criteria. However, the sections on cognitive and skill development benefits present compelling evidence, supported by appropriate sub-sections.The exploration of improved patient outcomes lacks concrete examples but effectively outlines potential benefits. The section on challenges and future directions raises relevant points, but suggestions for solutions could enhance practicality.Overall, the paper combines theoretical, empirical, and practical evidence, offering a well-rounded approach. It establishes the significance of VR-based surgical training and concludes with a concise summary of key findings.

If you want more tips on using AI tools , understanding plagiarism , and citing sources , make sure to check out some of our other articles with explanations, examples, and formats.

• Citing ChatGPT
• Best grammar checker
• Best paraphrasing tool
• ChatGPT in your studies
• Is ChatGPT trustworthy?
• Types of plagiarism
• Self-plagiarism
• Avoiding plagiarism
• Academic integrity
• Best plagiarism checker

Citing sources

• Citation styles
• In-text citation
• Citation examples
• Annotated bibliography

Yes, you can use ChatGPT to summarise text . This can help you understand complex information more easily, summarise the central argument of your own paper, or clarify your research question.

You can also use Scribbr’s free text summariser , which is designed specifically for this purpose.

Yes, you can use ChatGPT to paraphrase text to help you express your ideas more clearly, explore different ways of phrasing your arguments, and avoid repetition.

However, it’s not specifically designed for this purpose. We recommend using a specialised tool like Scribbr’s free paraphrasing tool , which will provide a smoother user experience.

No, having ChatGPT write your college essay can negatively impact your application in numerous ways. ChatGPT outputs are unoriginal and lack personal insight.

Furthermore, Passing off AI-generated text as your own work is considered academically dishonest . AI detectors may be used to detect this offense, and it’s highly unlikely that any university will accept you if you are caught submitting an AI-generated admission essay.

However, you can use ChatGPT to help write your college essay during the preparation and revision stages (e.g., for brainstorming ideas and generating feedback).

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Driessen, K. (2023, August 17). How to Write a Paper with ChatGPT | Tips & Examples. Scribbr. Retrieved 6 May 2024, from https://www.scribbr.co.uk/using-ai-tools/chatgpt-paper/

Is this article helpful?

Koen Driessen

Other students also liked, how to write good chatgpt prompts, chatgpt citations | formats & examples, ethical implications of chatgpt, still have questions.

ChatGPT in higher education - a synthesis of the literature and a future research agenda

• Open access
• Published: 02 May 2024

Cite this article

You have full access to this open access article

• Pritpal Singh Bhullar 1 ,
• Mahesh Joshi 2 &
• Ritesh Chugh   ORCID: orcid.org/0000-0003-0061-7206 3  

1571 Accesses

33 Altmetric

Explore all metrics

ChatGPT has emerged as a significant subject of research and exploration, casting a critical spotlight on teaching and learning practices in the higher education domain. This study examines the most influential articles, leading journals, and productive countries concerning citations and publications related to ChatGPT in higher education, while also shedding light on emerging thematic and geographic clusters within research on ChatGPT’s role and challenges in teaching and learning at higher education institutions. Forty-seven research papers from the Scopus database were shortlisted for bibliometric analysis. The findings indicate that the use of ChatGPT in higher education, particularly issues of academic integrity and research, has been studied extensively by scholars in the United States, who have produced the largest volume of publications, alongside the highest number of citations. This study uncovers four distinct thematic clusters (academic integrity, learning environment, student engagement, and scholarly research) and highlights the predominant areas of focus in research related to ChatGPT in higher education, including student examinations, academic integrity, student learning, and field-specific research, through a country-based bibliographic analysis. Plagiarism is a significant concern in the use of ChatGPT, which may reduce students’ ability to produce imaginative, inventive, and original material. This study offers valuable insights into the current state of ChatGPT in higher education literature, providing essential guidance for scholars, researchers, and policymakers.

Similar content being viewed by others

Students’ voices on generative AI: perceptions, benefits, and challenges in higher education

What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education

Detection of gpt-4 generated text in higher education: combining academic judgement and software to identify generative ai tool misuse.

Avoid common mistakes on your manuscript.

1 Introduction

ChatGPT, or Chat Generative Pre-trained Transformer, is a popular generative Artificial Intelligence (AI) chatbot developed by OpenAI, employing natural language processing to deliver interactive human-like conversational experiences (Jeon et al., 2023 ; Angelis et al., 2023 ). ChatGPT utilises a pre-trained language learning model, derived from an extensive big-data corpus, to predict outcomes based on a given prompt (Crawford et al., 2023 ; Geerling et al., 2023 ; Li et al., 2023 ). Since its inception, ChatGPT has attracted widespread attention and popularity and has the potential to disrupt the education sector (Rana, 2023 ). According to a research survey of adults conducted by the Pew Research Centre, approximately 60% of adults in the United States and 78% of adults in Asia possess knowledge of ChatGPT; furthermore, men are more familiar with ChatGPT than women (Vogels, 2023 ). The study also found that among ethnic groups globally, individuals of Asian descent have the highest level of familiarity with AI-based large language models (LLMs).

People have found value in using ChatGPT for a wide range of purposes, including generating creative content, answering questions, providing explanations, offering suggestions, and even having casual conversations (Crawford et al., 2023 ; Throp, 2023 ; Wu et al., 2023 ). Furthermore, ChatGPT is an effective digital assistant for facilitating a thorough understanding of diverse and intricate subjects using simple and accessible language. Given these features, ChatGPT has the potential to bring about a paradigm shift in traditional methods of delivering instruction and revolutionise the future of education (Tlili et al., 2023 ). ChatGPT stands out as a promising tool for open education, enhancing the independence and autonomy of autodidactic learners through personalised support, guidance, and feedback, potentially fostering increased motivation and engagement (Firat, 2023 ). Its capabilities encompass facilitating complex learning, asynchronous communication, feedback provision, and cognitive offloading (Memarian & Doleck, 2023 ).

However, the rapid expansion of ChatGPT has also aroused apprehensions in the academic world, particularly after reports surfaced that the New York Department of Education had unexpectedly imposed a ban on access to the tool due to concerns about academic integrity violations (Sun et al., 2023 ; Neumann et al., 2023 ; Crawford et al., 2023 ). Students who use ChatGPT to produce superior written assignments may have an unfair advantage over peers who lack access (Farrokhnia et al., 2023 ; Cotton et al., 2023 ). Ethical concerns about the deployment of LLMs include the potential for bias, effects on employment, misuse and unethical deployment, and loss of integrity. However, there has been little research on the potential dangers that a sophisticated chatbot such as ChatGPT poses in the realm of higher education, particularly through the lens of a systematic literature review and bibliometric techniques.

In this light, this paper explores the literature on the application of ChatGPT in higher education institutions and the obstacles encountered in various disciplines from the perspectives of both faculty and students. The paper aims to analyse the current state of the field by addressing the following overarching research questions using bibliographic coupling, co-occurrence analysis, citation analysis, and co-authorship analysis:

What are the most influential articles in terms of citations in research related to ChatGPT in education?

What are the top journals and countries in terms of publication productivity related to the implications of ChatGPT in higher education institutions?

What are the emerging thematic clusters in research on the role and challenges of ChatGPT in teaching and learning in higher education institutions?

What are the geographic clusters in research on the role and challenges of ChatGPT in teaching and learning in higher education institutions?

2 Methodology

In conducting this study, publications on the impact of ChatGPT on various aspects of higher education institutions were systematically identified through an extensive search using Elsevier’s Scopus database, a comprehensive repository hosting over 20,000 globally ranked, peer-reviewed journals (Mishra et al., 2017 ; Palomo et al., 2017 ; Vijaya & Mathur, 2023 ). Scopus is a widely used database for bibliometric analyses and is considered one of the “largest curated databases covering scientific journals” (pg. 5116) in different subject areas (Singh et al., 2021 ). Widely acclaimed for its comprehensive coverage, Scopus has been extensively employed in bibliometric analyses across diverse disciplines, as evidenced by studies in capital structure theories, business research, entrepreneurial orientation and blockchain security (Bajaj et al., 2020 ; Donthu et al., 2020 ; Gupta et al., 2021 ; Patrício & Ferreira, 2020 ). Notably, despite the “extremely high” correlation between the Web of Science and Scopus databases, Scopus’s status as a superior and versatile data source for literature extraction is reinforced by its broader coverage of subject areas and categories compared to the narrower journal scope of Web of Science, facilitating scholars in locating literature most pertinent to the review area (Archambault et al., 2009 ; Paul et al., 2021 ). To ensure a systematic literature review, we adhered to the preferred reporting items for systematic reviews and meta-analysis (PRISMA) guidelines (Page et al., 2021 ) for the search, identification, selection, reading, and data extraction from the articles retrieved through the Scopus database (Fig.  1 ). Reliance on a single database is acceptable within the PRISMA framework (Moher et al., 2009 ).

Employing Boolean-assisted search queries, we aimed to capture a comprehensive range of topics related to ChatGPT’s impact on higher education institutions. Specific search queries were carefully selected to ensure a broad yet relevant search scope and included the following:

“ChatGPT and Teaching learning in universities” OR “Effect of ChatGPT in higher education institution” OR “ChatGPT and student assessment in higher education” OR “ChatGPT and academic integrity” OR “ChatGPT and teaching pedagogy in higher education institution” OR “ChatGPT and cheating student course assignment” OR “ChatGPT and teaching in higher education” OR “Implications of ChatGPT in higher education institutions” OR “ChatGPT and evaluation criteria in higher education institution” OR “ChatGPT in universities” OR “ChatGPT and student learnings. ”

The study includes papers published and included in the Scopus database on or before May 26, 2023 on the theme of ChatGPT and higher education. This timeframe was chosen to encompass the most recent and relevant literature available up to the point of data retrieval. Papers identified through the search queries underwent inclusion or exclusion based on predetermined criteria. Specifically, only papers published in journals were considered for this study, as these undergo a peer-review process and are subject to stringent selection criteria set by the journals, ensuring their quality and reliability. Papers in conference proceedings were excluded from the start of the search. Only papers written in English were included to maintain consistency and clarity, whereas others were excluded. Of the 48 research papers that were initially identified, 47 were ultimately selected for the bibliometric analysis, which was conducted using VOSviewer, a bibliometric analysis tool.

PRISMA Flowchart

From the identified pool of 47 articles, the analysis uncovered a nuanced distribution of research methodologies. Specifically, 11 studies were grounded in quantitative research methodologies, underscoring a quantitative focus within the literature. In contrast, a substantial majority of 31 articles embraced a qualitative framework, showcasing a diverse spectrum that included pure qualitative research, editorials, letters to the editor, and opinion pieces. Furthermore, the review brought to light four literature reviews, signifying a synthesis of existing knowledge, and identified one study that strategically employed a mixed-methods approach, blending both qualitative and quantitative research techniques.

To address the research questions, the selected publications underwent analysis using various bibliometric techniques. For the first and second research questions, citation analysis was employed. For the third and fourth research questions, bibliographic analysis was performed in VOSviewer software to generate clusters.

3 Findings and discussion

3.1 publication trend.

Information from the Scopus database indicates that academics began focusing on investigating various aspects of ChatGPT’s potential in higher education in 2022, as they published their findings in 2023. All academic articles in reputable publications in the Scopus database were published in 2023.

3.2 Citation analysis

Table  1 presents the top ten articles according to the number of citations. The number of articles increased significantly in 2023, consistent with the emerging nature and growing relevance of the topic. Exploring the ramifications of ChatGPT in higher education is a recent focal point for scholars, with numerous aspects warranting deeper investigation. The limited citation count, as anticipated, underscores that publications from 2023 are in the early stages of gaining visibility and recognition within the academic community.

The article by Thorp ( 2023 ), entitled “ChatGPT is fun, but not an author”, has received the highest number of citations (79). Thorp stresses the risks associated with implementing ChatGPT in the classroom. Although ChatGPT is an innovative AI tool, significant barriers remain to its implementation in the field of education. According to Thorp, using ChatGPT in academic writing is still inefficient. Thorp also expresses concerns about the rising prevalence of ChatGPT in the fabrication of scientific publications. The second most-cited work, “How Does ChatGPT Perform on the United States Medical Licensing Examination?” by Gilson and colleagues, has received 27 citations. Gilson et al. ( 2023 ) evaluated the accuracy, speed and clarity of ChatGPT’s responses to questions on the United States Medical Licensing Examination’s Step 1 and Step 2 tests. The text responses generated by ChatGPT were evaluated using three qualitative metrics: the logical justification of the chosen answer, the inclusion of information relevant to the question, and the inclusion of information extraneous to the question. The model attained a level of proficiency comparable to that of a third-year medical student. The study demonstrates the potential utility of ChatGPT as an interactive educational resource in the field of medicine to facilitate the acquisition of knowledge and skills. Third is Kasneci et al.’s article “ChatGPT for good? On opportunities and challenges of large language models for education”, with 13 citations. This paper examines the benefits and drawbacks of using language models in the classroom from the perspectives of both teachers and students. The authors find that these comprehensive language models can serve as a supplement rather than a replacement for classroom instruction. Each of the remaining top-ten articles mentioned the impact of ChatGPT on academic integrity in education and had received fewer than ten citations at the time of analysis.

Table  2 presents the top 10 journals in terms of the number of citations of publications related to the topic of ChatGPT in higher education. The journal Science , which published “ChatGPT is fun, but not an author,” was deemed most influential because it received the highest number of citations (79). JMIR Medical Education has published two articles that have been cited by 30 other research articles on the same topic. Journal of University Teaching and Learning Practise has published the most articles: three. Innovations in Education and Teaching International has published two articles on this topic, which together have been cited by six articles.

As shown in Table  3 , the majority of research articles pertaining to ChatGPT and higher education have originated from countries in Asia. Six of the top 10 countries for publishing articles on this topic are located in the Asian continent. However, the most influential studies in terms of citations have been produced by the United States, Germany, Australia, and the United Kingdom. Combined, these countries have received a total of 63 citations, with individual counts of 36, 17, 7, and 7, respectively. These four countries have 90% of the total citations of the top 10 most productive countries in the field of research on higher education perspectives on ChatGPT.

3.3 Bibliographic coupling

3.3.1 thematic clusters.

Four thematic clusters (TCs) were identified from the included research articles, as shown in Table  4 . VOSviewer was used to perform clustering based on bibliographic coupling. This method identifies relations between documents by examining publications that cite the same sources (Boyack & Klavans, 2010 ). VOSviewer clusters articles with a common knowledge base, assigning each publication to exactly one cluster. To implement this clustering technique, we assessed the co-occurrence of bibliographic references among articles within our dataset. Co-occurrence was determined by identifying shared references between articles, indicating a thematic connection (Boyack & Klavans, 2010 ). Articles sharing common references were considered to co-occur, enabling us to quantify the extent of thematic relationships based on the frequency of shared references. We identified and categorised thematic clusters within our dataset through the combined approach of VOSviewer clustering and co-occurrence analysis. This method typically results in a distribution of clusters, with a limited number of larger clusters and a more substantial number of smaller clusters.

The clusters were derived through an analysis of subordinate articles extracted from the Scopus database. VOSviewer systematically organised similar articles into distinct clusters based on the shared patterns of bibliographic references (Van Eck & Waltman, 2010 ). To ensure methodological transparency and robustness, we established clear criteria and parameters for clustering. Specifically, keywords with a minimum frequency ( n  = 5) were included in the analysis, and co-occurrence was calculated based on a pairwise comparison method. This systematic approach ensured the meaningful representation of thematic relationships within the dataset, guided by insights from previous literature (Jarneving, 2007 ). Using cluster analysis techniques, the articles were organised into cohesive groups characterised by the degree of thematic homogeneity guided by the nature of the research findings. This approach ensured a robust representation of the underlying thematic structure (Jarneving, 2007 ).

Furthermore, to mitigate the risk of subjective bias in thematic categorisation, a counter-coding approach was employed. A second researcher independently categorised thematic clusters identified by VOSviewer to assess inter-rater agreement. The level of agreement between the two researchers was assessed using Cohen’s kappa coefficient, ensuring the reliability and validity of the thematic classification process. The resulting kappa coefficient (0.69) indicated substantial agreement, suggesting a high level of agreement beyond what would be expected by chance alone (Gisev et al., 2013 ). Furthermore, the nomenclature assigned to each cluster was finalised based on the predominant research theme emerging from the analysis, providing a concise and informative label for each group.

TC1: ChatGPT and Academic Integrity: Cotton et al. ( 2023 ) describe ChatGPT as a double-edged sword that potentially threatens academic integrity. AI essay writing systems are programmed to churn out essays based on specific guidelines or prompts, and it can be difficult to distinguish between human and machine-generated writing. Thus, students could potentially use these systems to cheat by submitting essays that are not their original work (Dehouche, 2021 ). Kasneci et al. ( 2023 ) argue that effective pedagogical practices must be developed in order to implement large language models in classrooms. These skills include not only a deep understanding of the technology but also an appreciation of its constraints and the vulnerability of complex systems in general. In addition, educational institutions need to develop a clearly articulated plan for the successful integration and optimal use of big language models in educational contexts and teaching curricula. In addition, students need to be taught how to verify information through a teaching strategy emphasising critical thinking effectively. Possible bias in the generated output, the need for continuous human supervision, and the likelihood of unforeseen effects are just a few of the challenges that come with the employment of AI systems. Continuous monitoring and transparency are necessary to ensure academic integrity while using ChatGPT. Lim et al. ( 2023 ) report that ChatGPT poses academic integrity challenges for the faculty of higher education institutions, who must verify whether academic work (assignments, research reports, etc.) submitted by students is derived from the fresh perspective of data analysis or plagiarised and recycled (copying and pasting original work) by ChatGPT. ChatGPT may threaten student learning and classroom engagement if students have access to information and course assignments without assessing their integrity. Perkins ( 2023 ) also expresses concerns regarding academic integrity in the use of ChatGPT. Students are utilising ChatGPT to complete their course assignments without attribution rather than producing original work. Higher education institutions must establish clear boundaries regarding academic integrity and plagiarism in light of the growing utilisation of AI tools in academic and research settings. In addition, the challenges posed by AI essay writing systems like ChatGPT necessitate a multifaceted approach to safeguard academic integrity. Educational institutions should invest in comprehensive educational programs that not only teach students the ethical use of technology but also incorporate rigorous assessments of critical thinking skills. Additionally, integrating AI literacy into the curriculum, with a focus on understanding the limitations and potential biases of big language models, can empower students to discern between human and machine-generated content.

TC2: ChatGPT and Learning Environment: According to Crawford et al. ( 2023 ), increased stress levels and peer pressure among university students have created a favourable environment for the use of AI tools. ChatGPT provides enhanced educational opportunities for college-level students. It can help students identify areas they may have overlooked, offer guidance on additional reading materials, and enhance existing peer and teacher connections. In addition, ChatGPT can propose alternative methods of evaluating students beyond conventional assignments. Crawford et al. ( 2023 ) recommend providing practical assignments incorporating ChatGPT as a supplementary tool to reduce plagiarism. Su ( 2023 ) documents that ChatGPT can provide students with a personalised learning experience based on their specific needs. In addition, the ChatGPT platform can be used to create a virtual coaching system that offers prompt feedback to educators during their classroom evaluations. This approach fosters critical thinking and supports early childhood educators in refining their teaching methodologies to optimise interactive learning outcomes for students. Tang ( 2023b ) proposes that bolstering research integrity can be achieved by imposing restrictions on the utilisation of NLP-generated content in research papers. Additionally, the author advocates for transparency from researchers, emphasising the importance of explicitly stating the proportion of NLP-generated content incorporated in their papers. This recommendation prompts a critical examination of the role of AI-generated content in scholarly work, emphasising the importance of nurturing independent research and writing skills for both students and researchers.

TC3: ChatGPT and Student Engagement: Lee ( 2023 ) examines the ability of ChatGPT to provide an interactive learning experience and boost student engagement beyond textbook pedagogy. Iskender ( 2023 ) explains that ChatGPT provides a mechanism for students to generate and investigate diverse concepts expeditiously, thereby helping them engage in imaginative and evaluative thinking on specific subject matter. This approach has the potential to optimise time management for students and allow them to concentrate on more advanced cognitive activities. AI tools such as ChatGPT can potentially enhance the personalisation of learning materials by providing visual aids and summaries that can aid the learning process and significantly improve students’ competencies. Hence, leveraging ChatGPT in education can revolutionise learning by facilitating interactive experiences, nurturing imaginative thinking, and optimising time management for students.

TC4: ChatGPT and Scholarly Research: Ivanov and Soliman ( 2023 ) and Yan ( 2023 ) focus on the practical applications and implications of LLMs like ChatGPT in educational settings and scholarly research within the context of language learning, writing, and tourism. Yan’s investigation into ChatGPT’s application in second-language writing examines its effectiveness in addressing specific writing tasks at the undergraduate level. The findings underscore the nuanced balance between the strengths of ChatGPT and the inherent limitations in handling demanding academic writing tasks. Nevertheless, ChatGPT is also labelled as an ‘all-in-one’ solution for scholarly research and writing (Yan, 2023 ). In parallel, Ivanov and Soliman ( 2023 ) highlight that ChatGPT can assist scholars in the field of tourism research by composing preliminary literature reviews, substantiating their chosen methodologies, and creating visual aids such as tables and charts. Furthermore, the researchers outline that ChatGPT could provide valuable methodological ideas and insights by helping researchers generate questions and corresponding scales for inclusion in questionnaires. Hence, ChatGPT has the potential to become a valuable ally as a facilitator in academic writing processes and has the potential to transform the research workflow.

3.3.2 Geographic clusters

The results of the country-based bibliographic analysis are summarised in Table  5 . The present study utilised the prevailing research theme in the existing literature as a framework for categorising the countries into four distinct clusters on the basis of the number of documents published from different countries.

Cluster 1: Implications of ChatGPT for Student Examinations and Education : Cluster 1 is composed of five countries: Germany, Ireland, South Korea, Taiwan, and the United States. Researchers in these countries have emphasised the potential role of ChatGPT in higher education within the context of AI language models. Eleven research articles related to this theme were published by researchers based in the United States, the most in this cluster. The top three articles in Table  1 are from the United States. The study entitled “Opportunities and Challenges of Large Language Models for Education,” was authored by German researchers (Kasneci et al., 2023 ) and has been widely cited in the academic community (13 citations). The remaining studies were conducted by researchers from South Korea and Taiwan and focused on the impact of ChatGPT on the education sector and its associated opportunities and challenges. This cluster demonstrates that students could benefit greatly from using ChatGPT in performing various academic tasks, such as reviewing and revising their work, verifying the accuracy of homework answers, and improving the quality of their essays. It has also aided postgraduates whose first language is not English improve their writing, as ChatGPT can be instructed to rewrite a paragraph in a scholarly tone from scratch. The outcomes have demonstrated significant efficacy, thereby alleviating the cognitive load associated with translation for these students, enabling them to concentrate on the substance of their writing rather than the intricacies of composing in an unfamiliar language. To harness the potential benefits, future research could focus on developing targeted training programs for students and educators that emphasise the effective utilisation of ChatGPT to enhance not only academic tasks but also language proficiency for non-native English speakers, addressing both cognitive load and language intricacies.

Cluster 2: ChatGPT and Academic Integrity : Cluster 2 comprises research studies conducted by authors from Japan, Bangladesh, Hong Kong, Nigeria, Pakistan, UAE, the UK, Vietnam and the Netherlands. The most influential study in this cluster, “Unlocking the power of ChatGPT: A framework for applying Generative AI in education”, was authored by researchers from Hong Kong (Su & Yang, 2023 ). They document that ChatGPT can be used to respond to student inquiries, reducing the time and effort required of educators and allowing them to focus their resources on other activities, such as scholarly investigations. Farrokhnia et al. ( 2023 ) and Yeadon et al. ( 2023 ) state that ChatGPT can write scientific abstracts with fabricated data and essays that can evade detection by reviewers. According to Liebrenz et al. ( 2023 ), ChatGPT tends to produce erroneous and incoherent responses, thereby raising the potential for disseminating inaccurate information in scholarly literature. The higher-order cognitive abilities of ChatGPT are relatively low, especially in areas related to creativity, critical thinking, reasoning, and problem-solving. ChatGPT could reduce students’ motivation to explore topics independently, draw their own conclusions, and solve problems independently (Kasneci et al., 2023 ). Ibrahim et al. ( 2023 ) find that ChatGPT can engage students in their academic pursuits. ChatGPT can enhance the writing abilities of non-native English speakers to allow them to concentrate on higher-order cognitive processes. This technological development allows faculty members to allocate more attention to conceptualisation and writing rather than focusing on the mechanics of grammar and spelling. However, there is a debate among intellectuals regarding the implications of AI for content creation, with some asserting that it detracts from innovative content development. The possibility that ChatGPT threatens academic honesty by facilitating essay plagiarism is being acknowledged. In addition, in the absence of appropriate citations, this textual content may violate copyright regulations. Cotton et al. ( 2023 ) express concerns about the potential impact of ChatGPT on academic integrity and plagiarism. Their work corroborates Dehouche’s ( 2021 ) assertion that students may use ChatGPT to engage in academic dishonesty by submitting essays that are not their original work. According to Cotton et al. ( 2023 ), ChatGPT users have a competitive advantage over non-users and can achieve higher grades on their coursework assignments by utilising the AI-based language tool. They classify ChatGPT as a versatile instrument with the potential to pose a threat to academic integrity, noting that AI essay writing systems are specifically programmed to generate content based on specific parameters or prompts, thereby challenging the discernment between human-authored and machine-generated content. Distinguishing between the academic work produced by students and the content of ChatGPT when evaluating assignments is a significant challenge for faculty. It is recommended that academic staff continually monitor student assignments for academic misconduct infractions, coupled with transparent communication about the potential risks associated with AI-generated content.

Cluster 3: ChatGPT and Students’ Learning : Cluster 3 comprises Malaysia, China and Australia. This cluster mainly includes studies of the role of AI-based models in student learning. Researchers from Australia (Crawford et al., 2023 ; Lim et al., 2023 ; Lawrie, 2023 ; Li et al., 2023 ; Seth et al., 2023 ; Cingillioglu, 2023 ; Skavronskaya, 2023 ; and Johinke, 2023 ) have contributed the most (8 studies) to this cluster and put their weight behind the role of AI and student learning in various disciplines. One of the most influential papers, “Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators”, was authored by researchers from both Australia and Malaysia (Lim et al., 2023 ) and reflected on the role of AI in classroom learning and teaching. Rather than banning AI tools, the authors advocate for the productive use of these tools in classrooms to facilitate more engaging student learning. Another Australian study titled, “Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI)” (Crawford et al., 2023 ) highlights AI as an alternative path of learning for students. ChatGPT can promptly evaluate students’ assignments and help them identify areas of weakness. Educators have the option to provide innovative assessments to their students instead of adhering solely to conventional assessments. ChatGPT can augment pedagogical approaches, evaluation structures, and the comprehensive educational milieu by reinforcing the trilateral association among instructors, learners, and technology. The implementation of ChatGPT can provide students with a personalised and interactive learning and research experience facilitated by virtual tutors and customised recommendations. In light of the research in this cluster, the integration of ChatGPT into education should inspire a paradigm shift towards a more dynamic and personalised learning environment. Institutions can explore strategic partnerships with AI researchers to develop context-specific applications of ChatGPT that cater to diverse educational needs, promoting a symbiotic relationship between human instructors, students, and technology for an enriched learning experience.

Cluster 4: ChatGPT and Field-specific Research : This cluster includes research by authors in Asian and European countries (India, Oman, Bulgaria and New Zealand) that has emphasised the potential role of ChatGPT in the medical and tourism industries. Authors from India explored the role of ChatGPT in the medical field (Seetharaman, 2023 ; Subramani et al., 2023 ). Seetharaman ( 2023 ) reports that ChatGPT offers supplementary language assistance to students who are not proficient in English, enabling them to enhance their language proficiency and effectively communicate in English, the principal language of instruction in medical establishments. The ChatGPT platform has the potential to serve as a tool for medical students to replicate patient interactions in a simulated environment, such as accurately obtaining medical histories and documenting symptoms. According to Subramani et al. ( 2023 ), ChatGPT is a highly efficient and user-friendly AI technology that can aid healthcare professionals in various aspects, such as diagnosis, critical decision-making, and devising appropriate treatment plans. ChatGPT has demonstrated impressive performance on medical exams, indicating its potential as a valuable resource for enhancing medical education and assessment (Subramani et al., 2023 ) and can support interdisciplinarity in tourism research (Nautiyal et al., 2023 ). Ivanov and Soliman ( 2023 ) note the potential of ChatGPT to serve as a digital instructor to provide students with enhanced and effective learning experiences and outcomes. Digital instructors can impart knowledge in diverse languages and thus can be used to educate individuals of varying nationalities and backgrounds in the field of tourism. Furthermore, LLM-based chatbots, including ChatGPT, can assess written assignments and provide direction on linguistic proficiency, syntax, and composition, ultimately enhancing students’ scholarly writing proficiency. In exploring the intersection of ChatGPT with medical education, institutions can pioneer innovative approaches by using the platform to create immersive, simulated patient interactions that go beyond language assistance, allowing medical students to practice nuanced skills such as medical history gathering and symptom documentation. Simultaneously, leveraging ChatGPT as a versatile digital instructor offers a unique opportunity to provide cross-cultural and multilingual education, contributing to a more inclusive and globally competent workforce within the tourism industry.

3.4 Challenges of ChatGPT in higher education

In addition to some previously mentioned challenges, such as the potential for plagiarism, the investigation also identified other key challenges in implementing ChatGPT within the context of higher education’s teaching and learning environment. Wu and Yu ( 2023 ) found that the benefits of AI-based ChatGPT are more in higher education as compared to primary and secondary education. The study also reported that the novelty effects of AI chatbots may enhance learning outcomes in brief interventions, but their efficacy diminishes in longer interventions.

First, the implementation of ChatGPT within the educational context engenders learning impediments. In the absence of adequate monitoring and regulation, the technology could lead to human unintelligence and unlearning, but teachers will become more adaptive and create authentic assessments to enhance student learning (Alafnan et al., 2023 ; Lawrie, 2023 ). Second, the technology could be used in a manner that violates students’ privacy. If the model is not adequately secured, it could surreptitiously gather confidential data from students without their explicit awareness or authorisation (Kanseci, 2023). Third, the technology could facilitate discrimination against particular students. If the model is not trained on a dataset that accurately represents the entire student population, it has the potential to create disparities in educational access (Cingillioglu, 2023 ; Lin et al., 2023 ). Fourth, according to Ivanov and Soloman (2023), ChatGPT lacks access to real-time data. Therefore, its responses may be inconsequential, inaccurate, or outdated. The information provided in response to a specific query may also be insufficient. Gao et al. (2022) highlight the need for further investigation of the precision and scholarly authenticity of ChatGPT. Fifth, it may be difficult for ChatGPT to comprehend the context and subtleties of complex academic subjects and answer complex questions (Adetayo, 2023 ; Eysenbach, 2023 ; Neumann et al., 2023 ). The system can misinterpret inquiries, offer inadequate or inaccurate responses, or struggle to comprehend the fundamental purpose behind questions (Clark, 2023 ). In particular, ChatGPT may not have the requisite expertise in highly specialised or advanced subjects such as advanced mathematics or specific sciences. Hence, it may not deliver precise and accurate answers (Neumann et al., 2023 ; Fergus et al., 2023 ). Karaali ( 2023 ) claimed that the primary emphasis in the field of AI is currently directed towards the enhancement of advanced cognitive abilities and mental processes associated with quantitative literacy and quantitative reasoning. However, it is important to acknowledge that fundamental skills such as writing, critical thinking, and numeracy continue to serve as essential foundational components among students. Although AI is making significant progress in fundamental domains, it appears that students are experiencing a decline in performance in the context of fundamental skills. Consequently, NLP-based adaptive learner support and education require further investigation (Bauer et al., 2023 ).

In addressing the challenges of ChatGPT in education, educators need to adapt and develop authentic assessments that mitigate the risk of human unlearning, ensuring that technology enhances, rather than hinders, student learning experiences. Simultaneously, recognising the limitations of ChatGPT in comprehending the nuances of highly specialised subjects underscores the importance of balancing advancements in AI’s cognitive abilities with continued emphasis on fundamental skills like critical thinking, writing, and numeracy, urging a reevaluation of priorities in AI-driven educational research towards comprehensive learner support.

4 Conclusion, implications and agenda for future research

This study identified the most influential articles and top journals and countries in terms of citations and publication productivity related to ChatGPT in higher education, as well as highlighted emerging thematic clusters and geographic clusters in research on the role and challenges of ChatGPT in teaching and learning in higher education institutions. Articles on the topic of ChatGPT in higher education published up to May 2023 were identified by searching the Scopus database. Given the emergent nature of ChatGPT starting in late 2022, all the included articles were published in 2023. Thus, this specific research domain remains relatively unexplored. The findings of this analysis reveal that the United States is the most productive country in terms of research on the role of ChatGPT in higher education, especially relating to academic integrity and research. US researchers also emerged as the most influential in terms of number of citations in the literature. Our findings corroborate those of previous research (Crompton & Burke, 2023 ). However, 60% of the articles in our shortlisted literature emanated from Asian countries.

Four thematic clusters (academic integrity, student engagement, learning environment and research) were identified. Furthermore, the country-based bibliographic analysis revealed that research has focused on student examinations, academic integrity, student learning and field-specific research in medical and tourism education (Nautiyal et al., 2023 ; Subramani et al., 2023 ). Plagiarism is recognised as a major challenge that hinders students’ creativity, innovativeness and originality when using ChatGPT in their academic pursuits. To mitigate the potential drawbacks of using ChatGPT in educational and research settings, proactive measures should be taken to educate students and researchers alike on the nature of plagiarism, its negative impacts and academic integrity (Shoufan, 2023 ; Teixeira, 2023 ) Educators may ask students to provide a written acknowledgement of the authenticity of their assignments and their non-reliance on ChatGPT. Such an acknowledgement would discourage students from utilising ChatGPT in their academic and research endeavours and establish accountability for their academic pursuits. In addition, educators should develop authentic assessments that are ChatGPT-proof.

ChatGPT lacks emotional intelligence and empathy, both of which are crucial in effectively addressing the emotional and psychological dimensions of the learning process (Farrokhnia et al., 2023 ; Neumann et al., 2023 ). Higher education institutions may encounter challenges in using ChatGPT to deliver suitable assistance, comprehension, or direction to students needing emotional or mental health support. The significance of human interaction in learning cannot be overstated. Achieving a balance between using AI and the advantages of human guidance and mentorship is a persistent challenge that requires attention (Neumann et al., 2023 ; Rahman et al., 2023 ). Strzelecki ( 2023 ) observed in his research that behavioural intention and personal innovativeness are the two major determinants behind the adoption of ChatGPT among students.

4.1 Implications

The findings of the present study have numerous important implications. This study provides insight into the current state of ChatGPT in higher education and thus can serve as valuable guidance for academics, practitioners, and policymakers. The study’s findings contribute to the literature by providing new insights into the role of ChatGPT and strategies for mitigating its negative aspects and emphasising its positive attributes.

First, the implementation of AI in education can improve academic performance and student motivation, particularly by facilitating personalised learning. Educational institutions should monitor and regulate students’ use of such technologies proactively. Higher education institutions also ought to prioritise the training of their educators in effectively utilising AI technologies, including ChatGPT. Concurrently, it is imperative for these institutions to equip students with comprehensive academic integrity training, shedding light on the appropriate and inappropriate applications of AI tools like ChatGPT. This includes creating awareness about the potential consequences of utilising these technologies for dishonest practices. Furthermore, educational establishments need to urgently revisit and refine their academic integrity policies to address the evolving landscape shaped by the integration of artificial intelligence tools in various academic facets. This proactive approach will foster a learning environment that embraces technological advancements and upholds the principles of honesty and responsible use. Institutional regulations on accountability and transparency should guide the frameworks that govern the use of AI in the campus environment (Pechenkina, 2023 ; Sun & Hoelscher, 2023 ; Dencik & Sanchez-Monedero, 2022 ).

Second, faculty members must proactively replace traditional coursework with modern alternatives that foster elevated levels of critical thinking among students, as suggested by Zhai ( 2022 ). Educators and learners can augment the academic material produced by ChatGPT with their own insights and information obtained from credible scholarly resources (Emenike & Emenike, 2023 ).

Third, ChatGPT should not be considered a threat to the education sector but a supplementary tool for human instruction that can enhance teaching and learning. It is imperative to acknowledge that the vital role of human educators cannot be replaced (Karaali, 2023 ) Moreover, ChatGPT can potentially enhance the accessibility and inclusivity of higher education. Alternative formats, linguistic support, and individualised explanations can help students who are studying English as a second language, are not native English speakers, or have other unique learning needs. Furthermore, Alnaqbi and Fouda ( 2023 ) highlight the implications of AI in evaluating the teaching style of faculty in higher education by collecting the feedback of students through social media and ChatGPT.

Fourth, the faculty in higher education institutions could address ethical concerns by providing students with explicit and comprehensive guidelines about the prescribed structure of academic assignments (Cotton et al., 2023 ; Gardner & Giordano, 2023 ). This practice can facilitate the production of more cohesive assignments. In addition, teachers can use rubrics to assess assignments and blend automated and manual assessment methodologies to evaluate students’ comprehension of the subject matter (Cotton et al., 2023 ; Shoufan, 2023 ).

In summary, using ChatGPT is recommended for enhancing creativity, refining writing proficiency, and improving research abilities. Nonetheless, it is crucial to emphasise that ChatGPT should not be employed as a substitute for critical thinking and producing original work. While it serves as a valuable tool for augmentation, upholding the integrity of independent thought and authentic content creation in academic endeavours is essential.

4.2 Limitations

The present study acknowledges several limitations. Firstly, the reliance on Scopus as the primary data source for bibliometric analysis may have limitations in capturing the full landscape of relevant literature. Future research may consider incorporating additional databases like Web of Science to ensure a comprehensive assessment. Secondly, due to the English language restriction in the review, potentially relevant studies may have been omitted. Future research could enhance inclusivity by extending its scope to encompass papers written in languages other than English. Thirdly, the current study exclusively focused on journal articles. Expanding the scope to include diverse sources, such as conference proceedings or book chapters, could offer a more comprehensive overview.

Additionally, as a rapidly evolving field, literature published after our inclusion dates need capturing, and future studies should consider adjusting their inclusion criteria to accommodate the dynamic nature of the subject matter. Lastly, the specificity of the bibliometric data search, centred around terms like ChatGPT, AI, higher education, and academic integrity, may have excluded certain relevant articles. Future studies should consider employing more generalised search parameters to encompass synonyms associated with these terms.

4.3 Future scope

The findings of the study suggest new avenues for future research. The effectiveness of evaluation criteria for assessments incorporating ChatGPT-generated text needs to be investigated. Specifically, the appropriate level of ChatGPT-produced text that students may use in academic tasks or assessments has not been established. Research on the ethical implications of using AI tools such as ChatGPT in higher education is also needed. Issues pertaining to data confidentiality, bias, and transparency in algorithms used for decision-making remain to be addressed. Feasible approaches for mitigating the excessive reliance of scholars and learners on ChatGPT or similar AI models are needed. Researchers could also explore the implementation of verification processes that go beyond traditional plagiarism detection methods, accounting for the unique challenges posed by AI systems. Future research in this domain could focus on establishing guidelines and best practices for the integration of AI tools like ChatGPT in academic settings, ensuring a balance between technological innovation and the preservation of academic rigour. Finally, the literature on ChatGPT in higher education has largely focused on the medical and tourism sectors. Future researchers must explore applications of ChatGPT in other disciplines.

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author upon reasonable request.

Adetayo, A. J. (2023). ChatGPT and librarians for reference consultations. Internet Reference Services Quarterly , 27 (3), 131–147.

Article   Google Scholar  

AlAfnan, M. A., Dishari, S., Jovic, M., & Lomidze, K. (2023). ChatGPT as an educational tool: Opportunities, challenges, and recommendations for communication, business writing, and composition courses. Journal of Artificial Intelligence and Technology , 3 (2), 60–68. https://doi.org/10.37965/jait.2023.0184 .

Alnaqbi, N. M., & Fouda, W. (2023). Exploring the role of ChatGPT and social media in enhancing student evaluation of teaching style in higher education using Neutrosophic sets. International Journal of Neutrosophic Science , 20 (4), 181–190.

Angelis, L. D., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023). ChatGPT and the rise of large language models: The new AI-driven infodemic threat in public health. Frontiers in Public Health , 11 , 1–8.

Archambault, E., Campbell, D., Gingras, Y., & Larivière, V. (2009). Comparing bibliometric statistics obtained from the web of Science and Scopus. Journal of the American Society for Information Science and Technology , 60 (7), 1320–1326.

Bajaj, Y., Kashiramka, S., & Singh, S. (2020). Application of capital structure theories: A systematic review. Journal of Advances in Management Research , 18 (2), 173–199. https://doi.org/10.1108/JAMR-01-2020-001 .

Bauer, E., Greisel, M., Kuznetsov, I., Berndt, M., Kollar, I., Dresel, M., Fischer, M. R., & Fischer, F. (2023). Using natural language processing to support peer-feedback in the age of artificial intelligence: A cross-disciplinary framework and a research agenda. British Journal of Educational Technology , 54 (5), 1222–1245.

Boyack, K. W., & Klavans, R. (2010). Co-citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately? Journal of the American Society for Information Science and Technology , 61 (12), 2389–2404.

Cingillioglu, I. (2023). Detecting AI-generated essays: The ChatGPT challenge. International Journal of Information and Learning Technology , 40 (3), 259–268. https://doi.org/10.1108/IJILT-03-2023-0043 .

Clark, T. M. (2023). Investigating the Use of an Artificial Intelligence Chatbot with General Chemistry exam questions. Journal of Chemical Education , 100 (5), 1905–1916.

Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International , 1–13. https://doi.org/10.1080/14703297.2023.2190148 .

Crawford, J., Cowling, M., & Allen, K. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching & Learning Practice , 20 (3), 1–21. https://doi.org/10.53761/1.20.3.02 .

Crompton, H., & Burke, D. (2023). Artificial intelligence and higher education: The state of the field. International Journal of Educational Technology in Higher Education , 20 , 1–22. https://doi.org/10.1186/s41239-023-00392-8 .

Dehouche, N. (2021). Plagiarism in the age of massive generative pre-trained transformers (GPT-3). Ethics in Science and Environmental Politics , 2 , 17–23. https://doi.org/10.3354/esep00195 .

Dencik, L., & Sanchez-Monedero, J. (2022). Data justice. Internet Policy Review , 11 (1), 1–16. https://doi.org/10.14763/2022.1.1615 .

Donthu, N., Kumar, S., & Pattnaik, D. (2020). Forty-five years of Journal of Business Research : A bibliometric analysis. Journal of Business Research , 109 , 1–14. https://doi.org/10.1016/j.jbusres.2019.10.039 .

Emenike, M., & Emenike, B. (2023). Was this title generated by ChatGPT? Considerations for artificial intelligence text-generation software programs for chemists and chemistry educators. Journal of Chemical Education , 100 (4), 1413–1418. https://doi.org/10.1021/acs.jchemed.3c00063 .

Eysenbach, G. (2023). The role of ChatGPT, generative language models, and artificial intelligence in medical education: A conversation with ChatGPT and a call for papers. JMIR Medical Education , 6 (9), 1–13. https://doi.org/10.2196/46885 .

Farrokhnia, M., Banihashem, K. S., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International , 1–15. https://doi.org/10.1080/14703297.2023.2195846 .

Fergus, S., Botha, M., & Ostovar, M. (2023). Evaluating academic answers generated using ChatGPT. Journal of Chemical Education , 100 (4), 1672–1675. https://doi.org/10.1021/acs.jchemed.3c00087 .

Firat, M. (2023). How chat GPT can transform autodidactic experiences and open education. Department of Distance Education, Open Education Faculty , Anadolu University, 1–6. https://doi.org/10.31219/osf.io/9ge8m .

Gardner, D. E., & Giordano, A. E. (2023). The challenges and value of undergraduate oral exams in the physical chemistry classrooms: A useful tool in the assessment toolbox. Journal of Chemical Education , 100 (5), 1705–1709.

Geerling, W., Mateer, G. D., Wooten, J., & Damodaran, N. (2023). ChatGPT has aced the test of understanding in college economics: Now what? The American Economist , 68 (2), 233–245.

Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023). How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Medical Education , 9 , 1–9. https://doi.org/10.2196/45312 .

Gisev, N., Bell, J. S., & Chen, T. F. (2013). Interrater agreement and interrater reliability: Key concepts, approaches, and applications. Research in Social and Administrative Pharmacy , 9 (3), 330–338.

Gupta, R., Pandey, R., & Sebastian, V. J. (2021). International entrepreneurial orientation (IEO): A bibliometric overview of scholarly research. Journal of Business Research , 125 , 74–88. https://doi.org/10.1016/j.jbusres.2020.12.00 .

Ibrahim, H., Asim, R., & Zaffar, F. (2023). Rethinking homework in the age of artificial intelligence. Intelligent Systems , 38 (2), 24–27.

Google Scholar  

Iskender, A. (2023). Holy or Unholy? Interview with Open AI’s ChatGPT. European Journal of Tourism Research , 34 , 3414. https://doi.org/10.54055/ejtr.v34i.3169 .

Ivanov, S., & Soliman, M. (2023). Game of algorithms: ChatGPT implications for the future of tourism education and research. Journal of Tourism Futures , 9 (2), 214–221.

Jarneving, B. (2007). Bibliographic coupling and its application to research-front and other core documents. Journal of Informetrics , 1 (4), 287–307.

Jeon, J., Lee, S., & Cho, S. (2023). A systematic review of research on speech-recognition chatbots for language learning: Implications for future directions in the era of large language models. Interactive Learning Environment , 1–19. https://doi.org/10.1080/10494820.2023.2204343 .

Johinke, R., Cummings, R., & Di Lauro, F. (2023). Reclaiming the technology of higher education for teaching digital writing in a post-pandemic world. Journal of University Teaching and Learning Practice , 20 (2), 1–18. https://doi.org/10.53761/1.20.02.01 .

Karaali, G. (2023). Artificial Intelligence, Basic skills, and quantitative literacy. Numeracy , 16 (1), 1–22. https://doi.org/10.5038/1936-4660.16.1.1438 .

Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences , 103 (2023), 102274–102271. https://doi.org/10.1016/j.lindif.2023.102274 .

Lawrie, G. (2023). Establishing a delicate balance in the relationship between artificial intelligence and authenticate assessment in student learning. Chemistry Education Research and Practice , 24 (2), 392–393.

Lee, H. (2023). The rise of ChatGPT: Exploring its potential in medical education. Anatomical Science Education , 1–6. https://doi.org/10.1002/ase.2270 .

Li, Y., Sha, L., Yan, L., Lin, J., Raković, M., Galbraith, K., Lyons, K., Gašević, D., & Chen, G. (2023). Can large language models write reflectively. Computer and Education: Artificial Intelligence , 4 (2023), 100140–100141.

Liebrenz, M., Schleifer, R., Buadze, A., Bhugra, D., & Smith, A. (2023). Generating scholarly content with ChatGPT: Ethical challenges for medical publishing. Lancet Digital Health , 5 (3), e105–106. https://doi.org/10.1016/S2589-7500(23)00019-5 .

Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education , 21 (2), 1–13.

Lin, C. C., Huang, A. Y. Q., Stephen, J. H., & Yang (2023). A review of AI-Driven conversational chatbots implementation methodologies and challenges (1999–2022). Sustainability , 15 (5), 4012. https://doi.org/10.3390/su15054012 .

Memarian, B., & Doleck, T. (2023). ChatGPT in education: Methods, potentials and limitations. Computers in Human Behavior: Artificial Humans , 100022 , 1–11.

Mishra, D., Gunasekaran, A., Papadopoulos, T., & Hazen, B. (2017). Green supply chain performance measures: A review and bibliometric analysis. Sustainable Production and Consumption , 10 , 85–99. https://doi.org/10.1016/j.spc.2017.01.003

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine , 151 (4), 264–269.

Nautiyal, R., Albrecht, J. N., & Nautiyal, A. (2023). ChatGPT and tourism academia. Annals of Tourism Research , 99 , 1–3.

Neumann, M., Rauschenberger, M., & Schon, E-M. (2023). 2023 IEEE/ACM 5th International Workshop on Software Engineering Education for the Next Generation (SEENG), Melbourne, Australia, 2023, pp. 29–32, https://doi.org/10.1109/SEENG59157.2023.00010 .

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., … & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ , 372 (71), 1–9. https://doi.org/10.1136/bmj.n71

Palomo, J., Figueroa-Domecq, C., & Laguna, P. (2017). Women, peace and security state-of-art: A bibliometric analysis in social sciences based on SCOPUS database. Scientometrics , 113 (1), 123–148. https://doi.org/10.1007/s11192-017-2484-x .

Patrício, L. D., & Ferreira, J. J. (2020). Blockchain security research: Theorizing through bibliographic-coupling analysis. Journal of Advances in Management Research , 18 (1), 1–35. https://doi.org/10.1108/JAMR-04-2020-0051 .

Paul, J., Lim, W. M., O’Cass, A., Hao, A. W., & Bresciani, S. (2021). Scientific procedures and rationales for systematic literature reviews (SPAR-4-SLR). International Journal of Consumer Studies . https://doi.org/10.1111/ijcs.12695 .

Pechenkina, K. (2023). Artificial intelligence for good? Challenges and possibilities of AI in higher education from a data justice perspective. In L. Czerniewicz, & C. Cronin (Eds.), Higher education for good: Teaching and learning futures (#HE4Good) . Open Book.

Perkins, M. (2023). Academic integrity considerations of AI large Language models in the post-pandemic era: ChatGPT and beyond. Journal of University Teaching & Learning Practice , 20 (2), 1–26. https://doi.org/10.53761/1.20.02.07 .

Rahman, M., Mostafizer, & Yutaka Watanobe. (2023). ChatGPT for Education and Research: Opportunities, threats, and strategies. Applied Sciences , 13 (9), 1–21. https://doi.org/10.3390/app13095783 .

Rana, S. (2023). AI and GPT for Management scholars and practitioners: Guidelines and implications. FIIB Business Review , 12 (1), 7–9.

Seetharaman, R. (2023). Revolutionising medical education: Can ChatGPT boost subjective learning and expression? Journal of Medical System , 47 (61), 1–4. https://doi.org/10.1007/s10916-023-01957-w .

Seth, I., Bulloch, G., & Lee, C. H. A. (2023). Redefining academic integrity, authorship, and innovation: The impact of ChatGPT on surgical research. Annals of Surgical Oncology , 30 , 5284–5285. https://doi.org/10.1245/s10434-023-13642-w .

Shoufan, A. (2023). Exploring students’ perceptions of ChatGPT: Thematic analysis and follow-up survey. IEEE Access , 11, 38805–38818, 2023, https://doi.org/10.1109/ACCESS.2023.3268224 .

Singh, V. K., Singh, P., Karmakar, M., Leta, J., & Mayr, P. (2021). The journal coverage of web of Science, Scopus and dimensions: A comparative analysis. Scientometrics , 126 , 5113–5142.

Skavronskaya, L., Hadinejad, A., & Cotterell, D. (2023). Reversing the threat of artificial intelligence to opportunity: A discussion of ChatGPT in tourism education. Journal of Teaching in Travel & Tourism , 23 , 1–6. https://doi.org/10.1080/15313220.2023.2196658 .

Strzelecki, A. (2023). To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interactive Learning Environments , 1–15. https://doi.org/10.1080/10494820.2023.2209881 .

Su, J., & Yang, W. (2023). Unlocking the power of ChatGPT: A framework for applying generative AI in education. ECNU Review of Education , 6 (3), 355–366.

Subramani, M., Jallel, I., & Mohan, S. K. (2023). Evaluating the performance of ChatGPT in medical physiology university examination of phase I MBBS. Advances in Physical Education , 47 , 270–271.

Sun, G. H., & Hoelscher, S. H. (2023). The ChatGPT Storm and what Faculty can do. Nurse Educator , 48 , 119–124.

Tang, G. (2023a). Academic journals cannot simply require authors to declare that they used ChatGPT. Irish Journal of Medical Science , 192 (6), 3195–3196.

Tang, G. (2023b). Letter to editor: Academic journals should clarify the proportion of NLP-generated content in papers. Accountability in Research , 1–3. https://doi.org/10.1080/08989621.2023.2180359 .

Teixeira da Silva, J. A. (2023). How are authors’ contributions verified in the ICMJE model? Plant Cell Reports , 42 , 1529–1153. https://doi.org/10.1007/s00299-023-03022-9 .

Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science , 379 (6630), 313–313.

Tlili, A., Shehata, B., Adarkwah, M., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments , 10 (15), 1–24.

Van Eck, N., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics , 84 (2), 523–538.

Vijaya, V., & Mathur, H. P. (2023). A decade of donation-based crowdfunding: A bibliometric analysis using the SCOPUS Database. Purushartha , 15 (2), 32–51.

Vogels, E. A. (2023). A majority of Americans have heard of ChatGPT, but few have tried it themselves, https://www.pewresearch.org/short-reads/2023/05/24/a-majority-of-americans-have-heard-of-chatgpt-but-few-have-tried-it-themselves/ .

Wu, R., & Yu, Z. (2023). Do AI Chatbots improve students learning outcome? Evidence from a meta analysis. British Journal of Educational Technology . https://doi.org/10.1111/bjet.13334 .

Wu, T. Y., He, S. Z., Liu, J. P., Sun, S. Q., Liu, K., Han, Q. L., & Tang, Y. (2023). A brief overview of ChatGPT: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica , 10 (5), 1122–1136. https://doi.org/10.1109/JAS.2023.123618 .

Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies , 1–25. https://doi.org/10.1007/s10639-023-11742-4 .

Yeadon, W., Inyang, O-O., Mizouri, A., Peach, A., & Testrow, C. P. (May 2023). The death of the short-term physics essay in the coming AI revolution. Physics Education , 58 , 1–13.

Zhai, X. (2022). ChatGPT user experience: Implications for education. SSRN , 4312418 , 1–18.

Download references

Open Access funding enabled and organized by CAUL and its Member Institutions

Author information

Authors and affiliations.

University Business School, Maharaja Ranjit Singh Punjab Technical University, Punjab, India

Pritpal Singh Bhullar

Department of Financial Planning and Tax, School of Accounting, Information Systems and Supply Chain, RMIT University, Melbourne, Australia

Mahesh Joshi

School of Engineering and Technology, CML‑NET & CREATE Research Centres, Central Queensland University, Queensland, Australia

Ritesh Chugh

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ritesh Chugh .

Ethics declarations

Conflict of interest.

No Conflict of Interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Bhullar, P.S., Joshi, M. & Chugh, R. ChatGPT in higher education - a synthesis of the literature and a future research agenda. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12723-x

Download citation

Received : 04 October 2023

Accepted : 16 April 2024

Published : 02 May 2024

DOI : https://doi.org/10.1007/s10639-024-12723-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial Intelligence
• Generative AI
• Higher education
• Academic integrity
• Systematic review
• Bibliometric analysis
• Find a journal
• Publish with us
• Track your research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 01 May 2024

A critical assessment of using ChatGPT for extracting structured data from clinical notes

• Jingwei Huang   ORCID: orcid.org/0000-0003-2155-6107 1 ,
• Donghan M. Yang 1 ,
• Ruichen Rong 1 ,
• Kuroush Nezafati   ORCID: orcid.org/0000-0002-6785-7362 1 ,
• Colin Treager 1 ,
• Zhikai Chi   ORCID: orcid.org/0000-0002-3601-3351 2 ,
• Shidan Wang   ORCID: orcid.org/0000-0002-0001-3261 1 ,
• Xian Cheng 1 ,
• Yujia Guo 1 ,
• Laura J. Klesse 3 ,
• Guanghua Xiao 1 ,
• Eric D. Peterson 4 ,
• Xiaowei Zhan 1 &
• Yang Xie   ORCID: orcid.org/0000-0001-9456-1762 1  

npj Digital Medicine volume  7 , Article number:  106 ( 2024 ) Cite this article

3665 Accesses

76 Altmetric

Metrics details

• Non-small-cell lung cancer

Existing natural language processing (NLP) methods to convert free-text clinical notes into structured data often require problem-specific annotations and model training. This study aims to evaluate ChatGPT’s capacity to extract information from free-text medical notes efficiently and comprehensively. We developed a large language model (LLM)-based workflow, utilizing systems engineering methodology and spiral “prompt engineering” process, leveraging OpenAI’s API for batch querying ChatGPT. We evaluated the effectiveness of this method using a dataset of more than 1000 lung cancer pathology reports and a dataset of 191 pediatric osteosarcoma pathology reports, comparing the ChatGPT-3.5 (gpt-3.5-turbo-16k) outputs with expert-curated structured data. ChatGPT-3.5 demonstrated the ability to extract pathological classifications with an overall accuracy of 89%, in lung cancer dataset, outperforming the performance of two traditional NLP methods. The performance is influenced by the design of the instructive prompt. Our case analysis shows that most misclassifications were due to the lack of highly specialized pathology terminology, and erroneous interpretation of TNM staging rules. Reproducibility shows the relatively stable performance of ChatGPT-3.5 over time. In pediatric osteosarcoma dataset, ChatGPT-3.5 accurately classified both grades and margin status with accuracy of 98.6% and 100% respectively. Our study shows the feasibility of using ChatGPT to process large volumes of clinical notes for structured information extraction without requiring extensive task-specific human annotation and model training. The results underscore the potential role of LLMs in transforming unstructured healthcare data into structured formats, thereby supporting research and aiding clinical decision-making.

Similar content being viewed by others

Highly accurate protein structure prediction with AlphaFold

scGPT: toward building a foundation model for single-cell multi-omics using generative AI

Augmenting large language models with chemistry tools

Introduction.

Large Language Models (LLMs) 1 , 2 , 3 , 4 , 5 , 6 , such as Generative Pre-trained Transformer (GPT) models represented by ChatGPT, are being utilized for diverse applications across various sectors. In the healthcare industry, early applications of LLMs are being used to facilitate patient-clinician communication 7 , 8 . To date, few studies have examined the potential of LLMs in reading and interpreting clinical notes, turning unstructured texts into structured, analyzable data.

Traditionally, the automated extraction of structured data elements from medical notes has relied on medical natural language processing (NLP) using rule-based or machine-learning approaches or a combination of both 9 , 10 . Machine learning methods 11 , 12 , 13 , 14 , particularly deep learning, typically employ neural networks and the first generation of transformer-based large language models (e.g., BERT). Medical domain knowledge needs to be integrated into model designs to enhance performance. However, a significant obstacle to developing these traditional medical NLP algorithms is the limited existence of human-annotated datasets and the costs associated with new human annotation 15 . Despite meticulous ground-truth labeling, the relatively small corpus sizes often result in models with poor generalizability or make evaluations of generalizability impossible. For decades, conventional artificial intelligence (AI) systems (symbolic and neural networks) have suffered from a lack of general knowledge and commonsense reasoning. LLMs, like GPT, offer a promising alternative, potentially using commonsense reasoning and broad general knowledge to facilitate language processing.

ChatGPT is the application interface of the GPT model family. This study explores an approach to using ChatGPT to extract structured data elements from unstructured clinical notes. In this study, we selected lung cancer pathology reports as the corpus for extracting detailed diagnosis information for lung cancer. To accomplish this, we developed and improved a prompt engineering process. We then evaluated the effectiveness of this method by comparing the ChatGPT output with expert-curated structured data and used case studies to provide insights into how ChatGPT read and interpreted notes and why it made mistakes in some cases.

Data and endpoints

The primary objective of this study was to develop an algorithm and assess the capabilities of ChatGPT in processing and interpreting a large volume of free-text clinical notes. To evaluate this, we utilized unstructured lung cancer pathology notes, which provide diagnostic information essential for developing treatment plans and play vital roles in clinical and translational research. We accessed a total of 1026 lung cancer pathology reports from two web portals: the Cancer Digital Slide Archive (CDSA data) ( https://cancer.digitalslidearchive.org/ ) and The Cancer Genome Atlas (TCGA data) ( https://cBioPortal.org ). These platforms serve as public data repositories for de-identified patient information, facilitating cancer research. The CDSA dataset was utilized as the “training” data for prompt development, while the TCGA dataset, after removing the overlapping cases with CDSA, served as the test data for evaluating the ChatGPT model performance.

From all the downloaded 99 pathology reports from CDSA for the training data, we excluded 21 invalid reports due to near-empty content, poor scanning quality, or missing report forms. Seventy-eight valid pathology reports were included as the training data to optimize the prompt. To evaluate the model performance, 1024 pathology reports were downloaded from cBioPortal. Among them, 97 overlapped with the training data and were excluded from the evaluation. We further excluded 153 invalid reports due to near-empty content, poor scanning quality, or missing report forms. The invalid reports were preserved to evaluate ChatGPT’s handling of irregular inputs separately, and were not included in the testing data for accuracy performance assessment. As a result, 774 valid pathology reports were included as the testing data for performance evaluation. These valid reports still contain typos, missing words, random characters, incomplete contents, and other quality issues challenging human reading. The corresponding numbers of reports used at each step of the process are detailed in Fig. 1 .

Exclusions are accounted for due to reasons such as empty reports, poor scanning quality, and other factors, including reports of stage IV or unknown conditions.

The specific task of this study was to identify tumor staging and histology types which are important for clinical care and research from pathology reports. The TNM staging system 16 , outlining the primary tumor features (T), regional lymph node involvement (N), and distant metastases (M), is commonly used to define the disease extent, assign prognosis, and guide lung cancer treatment. The American Joint Committee on Cancer (AJCC) has periodically released various editions 16 of TNM classification/staging for lung cancers based on recommendations from extensive database analyses. Following the AJCC guideline, individual pathologic T, N, and M stage components can be summarized into an overall pathologic staging score of Stage I, II, III, or IV. For this project, we instructed ChatGPT to use the AJCC 7 th edition Cancer Staging Manual 17 as the reference for staging lung cancer cases. As the lung cancer cases in our dataset are predominantly non-metastatic, the pathologic metastasis (pM) stage was not extracted. The data elements we chose to extract and evaluate for this study are pathologic primary tumor (pT) and pathologic lymph node (pN) stage components, overall pathologic tumor stage, and histology type.

Overall Performance

Using the training data in the CDSA dataset ( n  = 78), we experimented and improved prompts iteratively, and the final prompt is presented in Fig. 2 . The overall performance of the ChatGPT (gpt-3.5-turbo-16k model) is evaluated in the TCGA dataset ( n  = 774), and the results are summarized in Table 1 . The accuracy of primary tumor features (pT), regional lymph node involvement (pN), overall tumor stage, and histological diagnosis are 0.87, 0.91, 0.76, and 0.99, respectively. The average accuracy of all attributes is 0.89. The coverage rates for pT, pN, overall stage and histological diagnosis are 0.97, 0.94, 0.94 and 0.96, respectively. Further details of the accuracy evaluation, F1, Kappa, recall, and precision for each attribute are summarized as confusion matrices in Fig. 3 .

Final prompt for information extraction and estimation from pathology reports.

For meaningful evaluation, the cases with uncertain values, such as “Not Available”, “Not Specified”, “Cannot be determined”, “Unknown”, et al. in reference and prediction have been removed. a Primary tumor features (pT), b regional lymph node involvement (pN), c overall tumor stage, and d histological diagnosis.

Inference and Interpretation

To understand how ChatGPT reads and makes inferences from pathology reports, we demonstrated a case study using a typical pathology report in this cohort (TCGA-98-A53A) in Fig. 4a . The left panel shows part of the original pathology report, and the right panel shows the ChatGPT output with estimated pT, pN, overall stage, and histology diagnosis. For each estimate, ChatGPT gives the confidence level and the corresponding evidence it used for the estimation. In this case, ChatGPT correctly extracted information related to tumor size, tumor features, lymph node involvement, and histology information and used the AJCC staging guidelines to estimate tumor stage correctly. In addition, the confidence level, evidence interpretation, and case summary align well with the report and pathologists’ evaluations. For example, the evidence for the pT category was described as “The pathology report states that the tumor is > 3 cm and < 5 cm in greatest dimension, surrounded by lung or visceral pleura.” The evidence for tumor stage was described as “Based on the estimated pT category (T2a) and pN category (N0), the tumor stage is determined to be Stage IB according to AJCC7 criteria.” It shows that ChatGPT extracted relevant information from the note and correctly inferred the pT category based on the AJCC guideline (Supplementary Fig. 1 ) and the extracted information.

a TCGA-98-A53A. An example of a scanned pathological report (left panel) and ChatGPT output and interpretation (right panel). All estimations and support evidence are consistent with the pathologist’s evaluations. b The GPT model correctly inferred pT as T2a based on the tumor’s size and involvement according to AJCC guidelines.

In another more complex case, TCGA-50-6590 (Fig. 4b ), ChatGPT correctly inferred pT as T2a based on both the tumor’s size and location according to AJCC guidelines. Case TCGA-44-2656 demonstrates a more challenging scenario (Supplementary Fig. 2 ), where the report only contains some factual data without specifying pT, pN, and tumor stage. However, ChatGPT was able to infer the correct classifications based on the reported facts and provide proper supporting evidence.

Error analysis

To understand the types and potential reasons for misclassifications, we performed a detailed error analysis by looking into individual attributes and cases where ChatGPT made mistakes, the results of which are summarized below.

Primary tumor feature (pT) classification

In total, 768 cases with valid reports and reference values in the testing data were used to evaluate the classification performance of pT. Among them, 15 cases were reported with unknown or empty output by ChatGPT, making the coverage rate 0.97. For the remaining 753 cases, 12.6% of pT was misclassified. Among these misclassification cases, the majority were T1 misclassified as T2 (67 out of 753 or 8.9%) or T3 misclassified as T2 (12 out of 753, or 1.6%).

In most cases, ChatGPT extracted the correct tumor size information but used an incorrect rule to distinguish pT categories. For example, in the case TCGA-22-4609 (Fig. 5a ), ChatGPT stated, “Based on the tumor size of 2.0 cm, it falls within the range of T2 category according to AJCC 7th edition for lung carcinoma staging manual.” However, according to the AJCC 7 th edition staging guidelines for lung cancer, if the tumor is more than 2 cm but less than 3 cm in greatest dimension and does not invade nearby structures, pT should be classified as T1b. Therefore, ChatGPT correctly extracted the maximum tumor dimension of 2 cm but incorrectly interpreted this as meeting the criteria for classification as T2. Similarly, for case TCGA-85-A4JB, ChatGPT incorrectly claimed, “Based on the tumor size of 10 cm, the estimated pT category is T2 according to AJCC 7th edition for lung carcinoma staging manual.” According to the AJCC 7 th edition staging guidelines, a tumor more than 7 cm in greatest dimension should be classified as T3.

a TCGA-22-4609 illustrates a typical case where the GPT model uses a false rule, which is incorrect by AJCC guideline. b Case TCGA-39-5028 shows a complex case where there exist two tumors and the GPT model only capture one of them. c Case TCGA-39-5016 reveals a case where the GPT model made a mistake for getting confused with domain terminology.

Another challenging situation arose when multiple tumor nodules were identified within the lung. In the case of TCGA-39-5028 (Fig. 5b ), two separate tumor nodules were identified: one in the right upper lobe measuring 2.1 cm in greatest dimension and one in the right lower lobe measuring 6.6 cm in greatest dimension. According to the AJCC 7 th edition guidelines, the presence of separate tumor nodules in a different ipsilateral lobe results in a classification of T4. However, ChatGPT classified this case as T2a, stating, “The pathology report states the tumor’s greatest diameter as 2.1 cm”. This classification would be appropriated if the right upper lobe nodule were a single isolated tumor. However, ChatGPT failed to consider the presence of the second, larger nodule in the right lower lobe when determining the pT classification.

Regional lymph node involvement (pN)

The classification performance of pN was evaluated using 753 cases with valid reports and reference values in the testing data. Among them, 27 cases were reported with unknown or empty output by ChatGPT, making the coverage rate 0.94. For the remaining 726 cases, 8.5% of pN was misclassified. Most of these misclassification cases were N1 misclassified as N2 (32 cases). The AJCC 7th edition staging guidelines use the anatomic locations of positive lymph nodes to determine N1 vs. N2. However, most of the misclassification cases were caused by ChatGPT interpreting the number of positive nodes rather than the locations of the positive nodes. One such example is the case TCGA-85-6798. The report states, “Lymph nodes: 2/16 positive for metastasis (Hilar 2/16)”. Positive hilar lymph nodes correspond to N1 classification according to AJCC 7th edition guidelines. However, ChatGPT misclassifies this case as N2, stating, “The pathology report states that 2 out of 16 lymph nodes are positive for metastasis. Based on this information, the pN category can be estimated as N2 according to AJCC 7th edition for lung carcinoma staging manual.” This interpretation is incorrect, as the number of positive lymph nodes is not part of the criteria used to determine pN status according to AJCC 7th edition guidelines. The model misinterpreted pN2 predictions in 22 cases due to similar false assertions.

In some cases, the ChatGPT model made classification mistakes by misunderstanding the locations’ terminology. Figure 5c shows a case (TCGA-39-5016) where the ChatGPT model recognized that “6/9 peribronchial lymph nodes involved, “ corresponding with classification as N1, but ChatGPT misclassified this case as N2. By AJCC 7th edition guidelines, N2 is defined as “Metastasis in ipsilateral mediastinal and/or subcarinal lymph node(s)”. The ChatGPT model did not fully understand that terminology and made misclassifications.

Pathology tumor stage

The overall tumor stage classification performance was evaluated using 744 cases with valid reports and reference values as stage I, II and III in the testing data. Among them, 18 cases were reported as unknown or empty output by ChatGPT making the coverage rate as 0.94. For the remaining 726 cases, 23.6% of the overall stage was misclassified. Since the overall stage depends on individual pT and pN stages, the mistakes could come from misclassification of pT or pN (error propagation) or applying incorrect inference rules to determine the overall stage from pT and pN (incorrect rules). Looking into the 56 cases where ChatGPT misclassified stage II as stage III, 22 cases were due to error propagation, and 34 were due to incorrect rules. Figure 6a shows an example of error propagation (TCGA-MP-A4TK). ChatGPT misclassified the pT stage from T2a to T3, and then this mistake led to the incorrect classification of stage IIA to stage IIIA. Figure 6b illustrates a case (TCGA-49-4505) where ChatGPT made correct estimation of pT and pN but made false prediction about tumor stage by using a false rule. Among the 34 cases affected by incorrect rules, ChatGPT mistakenly inferred tumor stage as stage III for 26 cases where pT is T3 and pN is N0, respectively. For example, for case TCGA-55-7994, ChatGPT provided the evidence as “Based on the estimated pT category (T3) and pN category (N0), the tumor stage is determined to be Stage IIIA according to AJCC7 criteria”. According to AJCC7, tumors with T3 and N0 should be classified as stage IIB. Similarly, error analysis for other tumor stages shows that misclassifications come from both error propagation and applying false rules.

a Case TCGA-MP-A4TK: An example of typical errors GPT made in the experiments, i.e. GPT took false rule and further led to faulty propagation. b Case TCGA-49-4505: The GPT model made false estimation of Stage IIIA with a false rule, although it made correct inference with T2b and N1.

Histological diagnosis

The classification performance of histology diagnosis was evaluated using 762 cases with valid reports and reference values in the testing data. Among them, 17 cases were reported as either unknown or empty output by ChatGPT, making the coverage rate 0.96. For the remaining 745 cases, 6 ( < 1%) of histology types were misclassified. Among the mistakes that ChatGPT made for histology diagnosis, ChatGPT misclassified 3 of them as “other” type and 3 cases of actual “other” type (neither adenocarcinomas nor squamous cell carcinomas) as 2 adenocarcinomas and 1 squamous cell carcinoma. In TCGA-22-5485, two tumors exist: one squamous cell carcinoma and another adenocarcinoma, which should be classified as the ‘other’ type. However, ChatGPT only identified and extracted information for one tumor. In the case TCGA-33-AASB, which is the “other” type of histology, ChatGPT captured the key information and gave it as evidence: “The pathology report states the histologic diagnosis as infiltrating poorly differentiated non-small cell carcinoma with both squamous and glandular features”. However, it mistakenly estimated this case as “adenocarcinoma”. In another case (TCGA-86-8668) of adenocarcinoma, ChatGPT again captured key information and stated as evidence, “The pathology report states the histologic diagnosis as Bronchiolo-alveolar carcinoma, mucinous” but could not tell it is a subtype of adenocarcinoma. Both cases reveal that ChatGPT still has limitations in the specific domain knowledge in lung cancer pathology and the capability of correcting understanding its terminology.

Analyzing irregularities

The initial model evaluation and prompt-response review uncovered irregular scenarios: the original pathology reports may be blank, poorly scanned, or simply missing report forms. We reviewed how ChatGPT responded to these anomalies. First, when a report was blank, the prompt contained only the instruction part. ChatGPT failed to recognize this situation in most cases and inappropriately generated a fabricated case. Our experiments showed that, with the temperature set at 0 for blank reports, ChatGPT converged to a consistent, hallucinated response. Second, for nearly blank reports with a few random characters and poorly scanned reports, ChatGPT consistently converged to the same response with increased variance as noise increased. In some cases, ChatGPT responded appropriately to all required attributes but with unknown values for missing information. Last, among the 15 missing report forms in a small dataset, ChatGPT responded “unknown” as expected in only 5 cases, with the remaining 10 still converging to the hallucinated response.

Reproducibility evaluation

Since ChatGPT models (even with the same version) evolve over time, it is important to evaluate the stability and reproducibility of ChatGPT. For this purpose, we conducted experiments with the same model (“gpt-3.5-turbo-0301”), the same data, prompt, and settings (e.g., temperature = 0) twice in early April and the middle of May of 2023. The rate of equivalence between ChatGPT estimations in April and May on key attributes of interest (pT, pN, tumor stage, and histological diagnosis) is 0.913. The mean absolute error between certainty degrees in the two experiments is 0.051. Considering the evolutionary nature of ChatGPT models, we regard an output difference to a certain extent as reasonable and the overall ChatGPT 3.5 model as stable.

Comparison with other NLP methods

In order to have a clear perspective on how ChatGPT’s performance stands relative to established methods, we conducted a comparative analysis of the results generated by ChatGPT with two established methods: a keyword search algorithm and a deep learning-based Named Entity Recognition (NER) method.

Data selection and annotation

Since the keyword search and NER methods do not support zero-shot learning and require human annotations on the entity level, we carefully annotated our dataset for these traditional NLP methods. We used the same training and testing datasets as in the prompt engineering for ChatGPT. The training dataset underwent meticulous annotation by experienced medical professionals, adhering to the AJCC7 standards. This annotation process involved identifying and highlighting all relevant entities and text spans related to stage, histology, pN, and pT attributes. The detailed annotation process for the 78 cases required a few weeks of full-time work from medical professionals.

Keyword search algorithm using wordpiece tokenizer

For the keyword search algorithm, we employed the WordPiece tokenizer to segment words into subwords. We compiled an annotated entity dictionary from the training dataset. To assess the performance of this method, we calculated span similarities between the extracted spans in the validation and testing datasets and the entries in the dictionary.

Named Entity Recognition (NER) classification algorithm

For the NER classification algorithm, we designed a multi-label span classification model. This model utilized the pre-trained Bio_ClinicalBERT as its backbone. To adapt it for multi-label classification, we introduced an additional linear layer. The model underwent fine-tuning for 1000 epochs using the stochastic gradient descent (SGD) optimizer. The model exhibiting the highest overall F1 score on the validation dataset was selected as the final model for further evaluation in the testing dataset.

Performance evaluation

We evaluated the performance of both the keyword search and NER methods on the testing dataset. We summarized the predicted entities/spans and their corresponding labels. In cases where multiple related entities were identified for a specific category, we selected the most severe entities as the final prediction. Moreover, we inferred the stage information for corpora lacking explicit staging information by aggregating details from pN, pT, and diagnosis, aligning with the AJCC7 protocol. The overall predictions for stage, diagnosis, pN, and pT were compared against the ground truth table to gauge the accuracy and effectiveness of our methods. The results (Supplementary Table S1 ) show that the ChatGPT outperforms WordPiece tokenizer and NER Classifier. The average accuracy for ChatGPT, WordPiece tokenizer, and NER Classifier are 0.89, 0.51, and 0.76, respectively.

Prompt engineering process and results

Prompt design is a heuristic search process with many elements to consider, thus having a significantly large design space. We conducted many experiments to explore better prompts. Here, we share a few typical prompts and the performance of these prompts in the training data set to demonstrate our prompt engineering process.

Output format

The most straightforward prompt without special design would be: “read the pathology report and answer what are pT, pN, tumor stage, and histological diagnosis”. However, this simple prompt would make ChatGPT produce unstructured answers varying in format, terminology, and granularity across the large number of pathology reports. For example, ChatGPT may output pT as “T2” or “pT2NOMx”, and it outputs histological diagnosis as “Multifocal invasive moderately differentiated non-keratinizing squamous cell carcinoma”. The free-text answers will require a significant human workload to clean and process the output from ChatGPT. To solve this problem, we used a multiple choice answer format to force ChatGPT to pick standardized values for some attributes. For example, for pT, ChatGPT could only provide the following outputs: “T0, Tis, T1, T1a, T1b, T2, T2a, T2b, T3, T4, TX, Unknown”. For the histologic diagnosis, ChatGPT could provide output in one of these categories: Lung Adenocarcinoma, Lung Squamous Cell Carcinoma, Other, Unknown. In addition, we added the instruction, “Please make sure to output the whole set of answers together as a single JSON file, and don’t output anything beyond the required JSON file,” to emphasize the requirement for the output format. These requests in the prompt make the downstream analysis of ChatGPT output much more efficient. In order to know the certainty degree of ChatGPT’s estimate and the evidence, we asked ChatGPT to provide the following 4 outputs for each attribute/variable: extracted value as stated in the pathology report, estimated value based on AJCC 7th edition for lung carcinoma staging manual, the certainty degree of the estimation, and the supporting evidence for the estimation. The classification accuracy of this prompt with multiple choice output format (prompt v1) in our training data could achieve 0.854.

Evidence-based inference

One of the major concerns for LLM is that the results from the model are not supported by any evidence, especially when there is not enough information for specific questions. In order to reduce this problem, we emphasize the use of evidence for inference in the prompt by adding this instruction to ChatGPT: “Please ensure to make valid inferences for attribute estimation based on evidence. If there is no available evidence provided to make an estimation, please answer the value as “Unknown.” In addition, we asked ChatGPT to “Include “comment” as the last key of the JSON file.” After adding these two instructions (prompt v2), the performance of the classification in the training data increased to 0.865.

Chain of thought prompting by asking intermediate questions

Although tumor size is not a primary interest for diagnosis and clinical research, it plays a critical role in classifying the pT stage. We hypothesize that if ChatGPT pays closer attention to tumor size, it will have better classification performance. Therefore, we added an instruction in the prompt (prompt v3) to ask ChatGPT to estimate: “tumor size max_dimension: [<the greatest dimension of tumor in Centimeters (cm)>, ‘Unknown’]” as one of the attributes. After this modification, the performance of the classification in the training data increased to 0.90.

Providing examples

Providing examples is an effective way for humans to learn, and it should have similar effects for ChatGPT. We provided a specific example to infer the overall stage based on pT and pN by adding this instruction: “Please estimate the tumor stage category based on your estimated pT category and pN category and use AJCC7 criteria. For example, if pT is estimated as T2a and pN as N0, without information showing distant metastasis, then by AJCC7 criteria, the tumor stage is “Stage IB”.” After this modification (prompt v4), the performance of the classification in the training data increased to 0.936.

Although we can further refine and improve prompts, we decided to use prompt v4 as the final model and apply it to the testing data and get the final classification accuracy of 0.89 in the testing data.

ChatGPT-4 performance

LLM evolves rapidly and OpenAI just released the newest GPT-4 Turbo model (GPT-4-1106-preview) in November 2023. To compare this new model with GPT-3.5-Turbo, we applied this newest GPT model GPT-4-1106 to analyze all the lung cancer pathology notes in the testing data. The classification result and the comparison with the GPT-3.5-Turbo-16k are summarized in Supplementary Table 1 . The results show that GPT-4-turbo performs better in almost every aspect; overall, the GPT-4-turbo model increases performance by over 5%. However, GPT-4-Turbo is much more expensive than GPT-3.5-Turbo. The performance of GPT-3.5-Turbo-16k is still comparable and acceptable. As such, this study mainly focuses on assessing GPT-3.5-Turbo-16k, but highlights the fast development and promise of using LLM to extract structured data from clinical notes.

Analyzing osteosarcoma data

To demonstrate the broader application of this method beyond lung cancer, we collected and analyzed clinical notes from pediatric osteosarcoma patients. Osteosarcoma, the most common type of bone cancer in children and adolescents, has seen no substantial improvement in patient outcomes for the past few decades 18 . Histology grades and margin status are among the most important prognostic factors for osteosarcoma. We collected pathology reports from 191 osteosarcoma cases (approved by UTSW IRB #STU 012018-061). Out of these, 148 cases had histology grade information, and 81 had margin status information; these cases were used to evaluate the performance of the GPT-3.5-Turbo-16K model and our prompt engineering strategy. Final diagnoses on grade and margin were manually reviewed and curated by human experts, and these diagnoses were used to assess ChatGPT’s performance. All notes were de-identified prior to analysis. We applied the same prompt engineering strategy to extract grade and margin information from these osteosarcoma pathology reports. This analysis was conducted on our institution’s private Azure OpenAI platform, using the GPT-3.5-Turbo-16K model (version 0613), the same model used for lung cancer cases. ChatGPT accurately classified both grades (with a 98.6% accuracy rate) and margin status (100% accuracy), as shown in Supplementary Fig. 3 . In addition, Supplementary Fig. 4 details a specific case, illustrating how ChatGPT identifies grades and margin status from osteosarcoma pathology reports.

Since ChatGPT’s release in November 2022, it has spurred many potential innovative applications in healthcare 19 , 20 , 21 , 22 , 23 . To our knowledge, this is among the first reports of an end-to-end data science workflow for prompt engineering, using, and rigorously evaluating ChatGPT in its capacity of batch-processing information extraction tasks on large-scale clinical report data.

The main obstacle to developing traditional medical NLP algorithms is the limited availability of annotated data and the costs for new human annotations. To overcome these hurdles, particularly in integrating problem-specific information and domain knowledge with LLMs’ task-agnostic general knowledge, Augmented Language Models (ALMs) 24 , which incorporate reasoning and external tools for interaction with the environment, are emerging. Research shows that in-context learning (most influentially, few-shot prompting) can complement LLMs with task-specific knowledge to perform downstream tasks effectively 24 , 25 . In-context learning is an approach of training through instruction or light tutorial with a few examples (so called few-shot prompting; well instruction without any example is called 0-shot prompting) rather than fine-tuning or computing-intensive training, which adjusts model weights. This approach has become a dominant method for using LLMs in real-world problem-solving 24 , 25 , 26 . The advent of ALMs promises to revolutionize almost every aspect of human society, including the medical and healthcare domains, altering how we live, work, and communicate. Our study shows the feasibility of using ChatGPT to extract data from free text without extensive task-specific human annotation and model training.

In medical data extraction, our study has demonstrated the advantages of adopting ChatGPT over traditional methods in terms of cost-effectiveness and efficiency. Traditional approaches often require labor-intensive annotation processes that may take weeks and months from medical professionals, while ChatGPT models can be fine-tuned for data extraction within days, significantly reducing the time investment required for implementation. Moreover, our economic analysis revealed the cost savings associated with using ChatGPT, with processing over 900 pathology reports incurring a minimal monetary cost (less than $10 using GPT 3.5 Turbo and less than $30 using GPT-4 Turbo). This finding underscores the potential benefits of incorporating ChatGPT into medical data extraction workflows, not only for its time efficiency but also for its cost-effectiveness, making it a compelling option for medical institutions and researchers seeking to streamline their data extraction processes without compromising accuracy or quality.

A critical requirement for effectively utilizing an LLM is crafting a high-quality “prompt” to instruct the LLM, which has led to the emergence of an important methodology referred to as “prompt engineering.” Two fundamental principles guide this process: firstly, the provision of appropriate context, and secondly, delivering clear instructions about subtasks and the requirements for the desired response and how it should be presented. For a single query for one-time use, the user can experiment with and revise the prompt within the conversation session until a satisfactory answer is obtained. However, prompt design can become more complex when handling repetitive tasks over many input data files using the OpenAI API. In these instances, a prompt must be designed according to a given data feed while maintaining the generality and coverage for various input data features. In this study, we found that providing clear guidance on the output format, emphasizing evidence-based inference, providing chain of thought prompting by asking for tumor size information, and providing specific examples are critical in improving the efficiency and accuracy of extracting structured data from the free-text pathology reports. The approach employed in this study effectively leverages the OpenAI API for batch queries of ChatGPT services across a large set of tasks with similar input data structures, including but not limited to pathology reports and EHR.

Our evaluation results show that the ChatGPT (gpt-3.5-turbo-16k) achieved an overall average accuracy of 89% in extracting and estimating lung cancer staging information and histology subtypes compared to pathologist-curated data. This performance is very promising because some scanned pathology reports included in this study contained random characters, missing parts, typos, varied formats, and divergent information sections. ChatGPT also outperformed traditional NLP methods. Our case analysis shows that most misclassifications were due to a lack of knowledge of detailed pathology terminology or very specialized information in the current versions of ChatGPT models, which could be avoided with future model training or fine-tuning with more domain-specific knowledge.

While our experiments reveal ChatGPT’s strengths, they also underscore its limitations and potential risks, the most significant being the occasional “hallucination” phenomenon 27 , 28 , where the generated content is not faithful to the provided source content. For example, the responses to blank or near-blank reports reflect this issue, though these instances can be detected and corrected due to convergence towards an “attractor”.

The phenomenon of ‘hallucination’ in LLMs presents a significant challenge in the field. It is important to consider several key factors to effectively address the challenges and risks associated with ChatGPT’s application in medicine. Since the output of an LLM depends on both the model and the prompt, mitigating hallucination can be achieved through improvements in GPT models and prompting strategies. From a model perspective, model architecture, robust training, and fine-tuning on a diverse and comprehensive medical dataset, emphasizing accurate labeling and classification, can reduce misclassifications. Additionally, enhancing LLMs’ comprehension of medical terminology and guidelines by incorporating feedback from healthcare professionals during training and through Reinforcement Learning from Human Feedback (RLHF) can further diminish hallucinations. Regarding prompt engineering strategies, a crucial method is to prompt the GPT model with a ‘chain of thought’ and request an explanation with the evidence used in the reasoning. Further improvements could include explicitly requesting evidence from input data (e.g., the pathology report) and inference rules (e.g., AJCC rules). Prompting GPT models to respond with ‘Unknown’ when information is insufficient for making assertions, providing relevant context in the prompt, or using ‘embedding’ of relevant text to narrow down the semantic subspace can also be effective. Harnessing hallucination is an ongoing challenge in AI research, with various methods being explored 5 , 27 . For example, a recent study proposed “SelfCheckGPT” approach to fact-check black-box models 29 . Developing real-time error detection mechanisms is crucial for enhancing the reliability and trustworthiness of AI models. More research is needed to evaluate the extent, impacts, and potential solutions of using LLMs in clinical research and care.

When considering using ChatGPT and similar LLMs in healthcare, it’s important to thoughtfully consider the privacy implications. The sensitivity of medical data, governed by rigorous regulations like HIPAA, naturally raises concerns when integrating technologies like LLMs. Although it is a less concern to analyze public available de-identified data, like the lung cancer pathology notes used in this study, careful considerations are needed for secured healthcare data. More secured OpenAI services are offered by OpenAI security portal, claimed to be compliant to multiple regulation standards, and Microsoft Azure OpenAI, claimed could be used in a HIPAA-compliant manner. For example, de-identified Osteosarcoma pathology notes were analyzed by Microsoft Azure OpenAI covered by the Business Associate Agreement in this study. In addition, exploring options such as private versions of these APIs, or even developing LLMs within a secure healthcare IT environment, might offer good alternatives. Moreover, implementing strong data anonymization protocols and conducting regular security checks could further protect patient information. As we navigate these advancements, it’s crucial to continuously reassess and adapt appropriate privacy strategies, ensuring that the integration of AI into healthcare is both beneficial and responsible.

Despite these challenges, this study demonstrates our effective methodology in “prompt engineering”. It presents a general framework for using ChatGPT’s API in batch queries to process large volumes of pathology reports for structured information extraction and estimation. The application of ChatGPT in interpreting clinical notes holds substantial promise in transforming how healthcare professionals and patients utilize these crucial documents. By generating concise, accurate, and comprehensible summaries, ChatGPT could significantly enhance the effectiveness and efficiency of extracting structured information from unstructured clinical texts, ultimately leading to more efficient clinical research and improved patient care.

In conclusion, ChatGPT and other LLMs are powerful tools, not just for pathology report processing but also for the broader digital transformation of healthcare documents. These models can catalyze the utilization of the rich historical archives of medical practice, thereby creating robust resources for future research.

Data processing, workflow, and prompt engineering

The lung cancer data we used for this study are publicly accessible via CDSA ( https://cancer.digitalslidearchive.org/ ) and TCGA ( https://cBioPortal.org ), and they are de-identified data. The institutional review board at the University of Texas Southwestern Medical Center has approved this study where patient consent was waived for using retrospective, de-identified electronic health record data.

We aimed to leverage ChatGPT to extract and estimate structured data from these notes. Figure 7a displays our process. First, scanned pathology reports in PDF format were downloaded from TCGA and CDSA databases. Second, R package pdftools, an optical character recognition tool, was employed to convert scanned PDF files into text format. After this conversion, we identified reports with near-empty content, poor scanning quality, or missing report forms, and those cases were excluded from the study. Third, the OpenAI API was used to analyze the text data and extract structured data elements based on specific prompts. In addition, we extracted case identifiers and metadata items from the TCGA metadata file, which was used to evaluate the model performance.

a Illustration of the use of OpenAI API for batch queries of ChatGPT service, applied to a substantial volume of clinical notes — pathology reports in our study. b A general framework for integrating ChatGPT into real-world applications.

In this study, we implemented a problem-solving framework rooted in data science workflow and systems engineering principles, as depicted in Fig. 7b . An important step is the spiral approach 30 to ‘prompt engineering’, which involves experimenting with subtasks, different phrasings, contexts, format specifications, and example outputs to improve the quality and relevance of the model’s responses. It was an iterative process to achieve the desired results. For the prompt engineering, we first define the objective: to extract information on TNM staging and histology type as structured attributes from the unstructured pathology reports. Second, we assigned specific tasks to ChatGPT, including estimating the targeted attributes, evaluating certainty levels, identifying key evidence of each attribute estimation, and generating a summary as output. The output was compiled into a JSON file. In this process, clinicians were actively formulating questions and evaluating the results.

Our study used the “gpt-3.5-turbo” model, accessible via the OpenAI API. The model incorporates 175 billion parameters and was trained on various public and authorized documents, demonstrating specific Artificial General Intelligence (AGI) capabilities 5 . Each of our queries sent to ChatGPT service is a “text completion” 31 , which can be implemented as a single round chat completion. All LLMs have limited context windows, constraining the input length of a query. Therefore, lengthy pathology reports combined with the prompt and ChatGPT’s response might exceed this limit. We used OpenAI’s “tiktoken” Python library to estimate the token count to ensure compliance. This constraint has been largely relaxed by the newly released GPT models with much larger context windows. We illustrate the pseudocode for batch ChatGPT queries on a large pathology report set in Supplementary Fig. 5 .

Model evaluation

We evaluated the performance of ChatGPT by comparing its output with expert-curated data elements provided in the TCGA structured data using the testing data set. Some staging records in the TCGA structured data needed to be updated; our physicians curated and updated those records. To mimic a real-world setting, we processed all reports regardless of data quality to collect model responses. For performance evaluation, we only used valid reports providing meaningful text and excluded the reports with near-empty content, poor scanning quality, and missing report forms, which were reported as irregular cases. We assessed the classification accuracy, F1, Kappa, recall, and precision for each attribute of interest, including pT, pN, overall stage, and histology types, and presented results as accuracy and confusion matrices. Missing data were excluded from the accuracy evaluation, and the coverage rate was reported for predicted values as ‘unknown’ or empty output.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

The lung cancer dataset we used for this study is “Pan-Lung Cancer (TCGA, Nat Genet2016)”, ( https://www.cbioportal.org/study/summary?id=nsclc_tcga_broad_2016 ) and the “luad” and “lusc” subsets from CDSA ( https://cancer.digitalslidearchive.org/ ). We have provided a reference regarding how to access the data 32 . We utilized the provided APIs to retrieve clinical information and pathology reports for the LUAD (lung adenocarcinoma) and LUSC (lung squamous cell carcinoma) cohorts. The pediatric data are the EHR data from UTSW clinic services. The data is available from the corresponding author upon reasonable request and IRB approval.

Code availability

All codes used in this paper were developed using APIs from OpenAI. The prompt for the API is available in Fig. 2 . Method-specific code is available from the corresponding author upon request.

Vaswani, A. et al. Attention is all you need. Adv. Neural Info. Processing Syst. 30 , (2017).

Devlin, J. et al. Bert: Pre-training of deep bidirectional transformers for language understanding . arXiv preprint arXiv:1810.04805, 2018.

Radford, A. et al. Improving language understanding by generative pre-training . OpenAI: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf (2018).

Touvron, H. et al. LLaMA: Open and efficient foundation language models . arXiv preprint arXiv:2302.13971 (2023).

OpenAi, GPT-4 Technical Report . arXiv:2303.08774: https://arxiv.org/pdf/2303.08774.pdf (2023).

Anil, R. et al. Palm 2 technical report . arXiv preprint arXiv:2305.10403 (2023).

Turner, B. E. W. Epic, Microsoft bring GPT-4 to EHRs .

Landi, H. Microsoft’s Nuance integrates OpenAI’s GPT-4 into voice-enabled medical scribe software .

Hao, T. et al. Health Natural Language Processing: Methodology Development and Applications. JMIR Med Inf. 9 , e23898 (2021).

Article   Google Scholar  

Pathak, J., Kho, A. N. & Denny, J. C. Electronic health records-driven phenotyping: challenges, recent advances, and perspectives. J. Am. Med. Inform. Assoc. 20 , e206–e211 (2013).

Article   PubMed   PubMed Central   Google Scholar  

Crichton, G. et al. A neural network multi-task learning approach to biomedical named entity recognition. BMC Bioinforma. 18 , 368 (2017).

Wang, J. et al. Document-Level Biomedical Relation Extraction Using Graph Convolutional Network and Multihead Attention: Algorithm Development and Validation. JMIR Med Inf. 8 , e17638 (2020).

Liu, Y. et al. Roberta: A robustly optimized BERT pretraining approach . arXiv preprint arXiv:1907.11692 (2019).

Rasmy, L. et al. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. npj Digit. Med. 4 , 86 (2021).

Wu, H. et al. A survey on clinical natural language processing in the United Kingdom from 2007 to 2022. npj Digit. Med. 5 , 186 (2022).

Amin, M. B. et al. AJCC cancer staging manual . 1024: Springer 2017.

Goldstraw, P. et al. The IASLC Lung Cancer Staging Project: Proposals for the Revision of the TNM Stage Groupings in the Forthcoming (Seventh) Edition of the TNM Classification of Malignant Tumours. J. Thorac. Oncol. 2 , 706–714 (2007).

Article   PubMed   Google Scholar  

Yang, D. M. et al. Osteosarcoma Explorer: A Data Commons With Clinical, Genomic, Protein, and Tissue Imaging Data for Osteosarcoma Research. JCO Clin. Cancer Inform. 7 , e2300104 (2023).

The Lancet Digital, H., ChatGPT: friend or foe? Lancet Digital Health . 5 , e102 (2023).

Nature, Will ChatGPT transform healthcare? Nat. Med. 29 , 505–506 (2023).

Patel, S. B. & Lam, K. ChatGPT: the future of discharge summaries? Lancet Digit. Health 5 , e107–e108 (2023).

Article   CAS   PubMed   Google Scholar  

Ali, S. R. et al. Using ChatGPT to write patient clinic letters. Lancet Digit. Health 5 , e179–e181 (2023).

Howard, A., Hope, W. & Gerada, A. ChatGPT and antimicrobial advice: the end of the consulting infection doctor? Lancet Infect. Dis. 23 , 405–406 (2023).

Mialon, G. et al. Augmented language models: a survey . arXiv preprint arXiv:2302.07842 (2023).

Brown, T. et al. Language Models are Few-Shot Learners . Curran Associates, Inc. (2020).

Wei, J. et al. Chain of thought prompting elicits reasoning in large language models . Adv Neural Inf Processing Syst 35 , 24824–24837 (2022).

Ji, Z. et al. Survey of Hallucination in Natural Language Generation. ACM Comput. Surv. 55 , 1–38 (2023).

Alkaissi, H. & S. I. McFarlane, Artificial Hallucinations in ChatGPT: Implications in Scientific Writing. Cureus , (2023).

Manakul, P. A. Liusie, & M. J. F. Gales, SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models . 2023.

Boehm, B. W. A spiral model of software development and enhancement. Computer 21 , 61–72 (1988).

OpenAi. OpenAI API Documentation . Available from: https://platform.openai.com/docs/guides/text-generation .

Gao, J. et al. Integrative analysis of complex cancer genomics and clinical profiles using the cBioPortal. Sci. Signal. 6 , 1–19 (2013).

Download references

Acknowledgements

This work was partially supported by the National Institutes of Health [P50CA70907, R35GM136375, R01GM140012, R01GM141519, R01DE030656, U01CA249245, and U01AI169298], and the Cancer Prevention and Research Institute of Texas [RP230330 and RP180805].

Author information

Authors and affiliations.

Quantitative Biomedical Research Center, Peter O’Donnell School of Public Health, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, USA 75390, USA

Jingwei Huang, Donghan M. Yang, Ruichen Rong, Kuroush Nezafati, Colin Treager, Shidan Wang, Xian Cheng, Yujia Guo, Guanghua Xiao, Xiaowei Zhan & Yang Xie

Department of Pathology, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, USA 75390, USA

Department of Pediatrics, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, USA 75390, USA

Laura J. Klesse

Department of Internal Medicine, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, USA 75390, USA

Eric D. Peterson

You can also search for this author in PubMed   Google Scholar

Contributions

J.H., Y.X., X.Z. and G.X. designed the study. X.Z., K.N., C.T. and J.H. prepared, labeled, and curated lung cancer datasets. D.M.Y., X.C., Y.G., L.J.K. prepared, labeled, and curated osteosarcoma datasets. Z.C. provided critical inputs as pathologists. Y.X., G.X., E.P. provided critical inputs for the study. J.H. implemented experiments with ChatGPT. R.R. and K.N. implemented experiments with N.L.P. J.H., Y.X., G.X. and S.W. conducted data analysis. Y.X., G.X., J.H., X.Z., D.M.Y. and R.R. wrote the manuscript. All co-authors read and commented on the manuscript.

Corresponding authors

Correspondence to Xiaowei Zhan or Yang Xie .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplemental figures and tables, reporting summary, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Huang, J., Yang, D.M., Rong, R. et al. A critical assessment of using ChatGPT for extracting structured data from clinical notes. npj Digit. Med. 7 , 106 (2024). https://doi.org/10.1038/s41746-024-01079-8

Download citation

Received : 24 July 2023

Accepted : 14 March 2024

Published : 01 May 2024

DOI : https://doi.org/10.1038/s41746-024-01079-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.