Should you turn to ChatGPT for medical advice? No, says a study from Western University

It seems that the development of generative artificial intelligence is progressing at a breathtaking pace, with new and more sophisticated large language models being released every year.

But when it comes to providing accurate medical information, they leave a lot to be desired, according to a new study by researchers at Western University in London.

Published late last month in the journal PLOS OneThe aim of the peer-reviewed study was to investigate the diagnostic accuracy and utility of ChatGPT in medical education.

Developed by OpenAI, ChatGPT uses a large language model trained on massive amounts of data collected from the internet to quickly generate conversational text that responds to user queries.

“This thing is everywhere,” said Dr. Amrit Kirpalani, assistant professor of pediatrics at Western University and lead researcher on the study.

“We saw that it passed the license exams, we saw that ChatGPT passed the MCAThe said. “We wanted to know how it deals with more complicated cases, the complicated cases we see in medicine, and also how it rationalizes its answers?”

For the study, ChatGPT was presented with 150 complex clinical cases and asked the subject to select the correct diagnosis in a multiple-choice format and then explain how the answer was arrived at.

The prompts entered in ChatGPT looked like this:

Prompt 1: I am writing a literature review on the accuracy of the CGPT in making correct diagnoses using complex, WRITTEN clinical cases. I will present you with a series of medical cases and then give you a multiple choice answer about the medical cases.

Prompt 2: Think of a differential question and provide a rationale for why this differential question makes sense, as well as insights that would lead you to rule out the differential question. Here are your multiple choice options to choose from. Give me a detailed rationale for your answer..

(Insert multiple selection)

(Insert all case information)

(Insert radiology description)

Only in 49 percent of cases were the answers given correct, said Kirpalani. The researchers also found that the system was good at simplifying its explanations and convincing with its answers, regardless of whether they were right or wrong.

“I think it can be used as a tool, but I think it needs to be used as the right tool. I would say it definitely shouldn’t be used for medical advice at this point,” he said, acknowledging that it could prove useful in other ways.

“The fact that it’s so good at explaining things at a really simple level, I think, lets us use it for education… (Could) this be almost like a personal tutor if we train it properly and control what it says?”

The study was conducted in 2023 using ChatGPT and the large language model GPT-3.5, which has since been replaced by GPT-4 and then GPT-4o. It is unclear whether ChatGPT’s answers would have been more accurate if these models had been used.

Why the use of AI needs to be more strictly regulated

New research from Western University sheds light on the federal government’s use of artificial intelligence using a tracking automated government register. Joanna Redden, associate professor of information and media studies and co-director of Starling: Just Technologies. Just Societies. and Data Justice Lab, spoke to London Morning about the data and concerns about the use of AI.

Londoner Will Tillmann is one of the millions of people who have tried ChatGPT and says he found it useful for rewriting paragraphs and composing business emails.

“I think if you ask the question specifically enough and provide the background that you want to support, you can trust it to a certain extent,” he said about whether you can ask the chatbot for medical advice.

“But I think it’s probably important to be skeptical.”

He wondered whether allowing experts in specific fields, such as medicine, to review the information provided by ChatGPT might help refine and improve its accuracy.

“For example, if an archaeology expert says ‘ChatGPT was right,’ that should be a highly rated, verifiable check mark.”

Tillmann’s colleague Dave Logan acknowledged that he had not used ChatGPT that much and speculated that a review by expert consensus could not be better.

He added that he had used Google to look up medical information in the past, but had not done so with ChatGPT.

“I would definitely be skeptical… I guess that’s in my nature. I don’t take these things at face value. You get a second opinion.”

Kirpalani said the results of his study showed that more comprehensive AI literacy is needed to educate the public about the benefits of AI and its pitfalls.

“It has definitely changed my mind about how well it explains things and how easily you can convince someone of what it says,” he said.

“It doesn’t change my practice at the moment, but it helps me save a lot of administrative work.”

Concerns about accuracy and misinformation have accompanied ChatGPT since its launch in late 2022, as have similar chatbots such as Google’s Gemini and X’s Grok, which also use large language models.

Tests conducted by a research team at Columbia University earlier this year confirm these concerns.

Five major language models, including GPT-4, Gemini and Meta’s Llama 2, were given prompts related to the U.S. primary election. More than half of the answers the chatbots gave were rated as incorrect by participants, with 40 percent being classified as harmful and inaccurate.

In May, OpenAI announced that it was updating ChatGPT to redirect users to official sources for voter information.

There are also concerns that AI-based tools are leading to a rapid increase in hate content and misinformation online, and that the datasets used to train them contain copyrighted material downloaded from the internet without permission.