Understanding the Limitations of ChatGPT for Cancer Treatment Recommendations
The Power of the Internet in Medical Education
For many patients, the Internet has become a powerful tool for self-education on medical topics. With the advent of ChatGPT, an artificial intelligence chatbot, patients now have access to a wealth of information. However, researchers at Brigham and Women’s Hospital have evaluated the consistency of the recommendations provided by ChatGPT for cancer treatment. Their findings, published in JAMA Oncology, revealed that in about a third of the cases, ChatGPT provided recommendations that did not align with clinical guidelines from the National Comprehensive Cancer Network (NCCN). This highlights the need to be aware of the limitations of this technology.
The Need for Collaboration between Patients and Doctors
While patients should feel empowered to learn about their medical conditions, it is crucial to remember that online resources should not be viewed in isolation. According to corresponding author Danielle Bitterman, MD, from the Department of Radiation Oncology and the General Mass Brigham’s Artificial Intelligence in Medicine (AIM) Program, ChatGPT’s responses can be very human-like and convincing. However, when it comes to clinical decision-making, each patient’s unique situation requires nuanced consideration. ChatGPT and other large language models may not necessarily provide the correct answer due to the complexity of individual cases.
To ensure the responsible incorporation of AI into healthcare delivery, Mass General Brigham, as one of the leading integrated academic health systems and innovation companies in the nation, is conducting rigorous research on new and emerging technologies. This research aims to inform the implementation of AI in care, workforce support, and administrative processes, thereby reshaping the continuity of patient care.
Evaluating ChatGPT’s Alignment with NCCN Guidelines
While medical decision-making can be influenced by various factors, Bitterman and colleagues specifically assessed the extent to which ChatGPT’s recommendations aligned with the NCCN guidelines. Physicians at institutions across the country rely on these guidelines for cancer treatment. The researchers focused on the three most common cancers—breast, prostate, and lung cancer—and asked ChatGPT to provide a treatment approach for each cancer based on disease severity. They used 26 unique diagnostic descriptions and four different prompts to generate a total of 104 scenarios.
The researchers discovered that nearly all of ChatGPT’s responses (98 percent) included at least one treatment approach that aligned with the NCCN guidelines. However, they also found that 34 percent of the responses included one or more non-matching recommendations, which were sometimes challenging to identify amidst the overall targeting. An inconsistent treatment recommendation was defined as one that was only partially correct, such as recommending surgery alone for locally advanced breast cancer without mentioning other therapy modalities. Significantly, full agreement on scoring occurred in only 62 percent of cases, emphasizing the complexity of the NCCN guidelines and the potential vagueness or difficulty to interpret ChatGPT’s results.
In some cases (12.5 percent), ChatGPT provided “hallucinations” or treatment recommendations that were completely missing from the NCCN guidelines. These recommendations included novel therapies or curative treatments for non-curable cancers. The authors emphasized that such misinformation can create incorrect expectations about treatment and potentially affect the doctor-patient relationship.
The Challenges of Language Models in Healthcare
As AI tools, including large language models (LLMs) like ChatGPT, continue to emerge in healthcare, it becomes crucial to distinguish between medical advice from a trained physician and information generated by LLMs. Shan Chen, MS, the first author of the study, highlights that users often rely on LLMs to learn about health-related topics, similar to how they use search engines like Google. However, it is essential to raise awareness that these language models are not equivalent to trained professionals, and their answers may not always be consistent or logical.
The study utilized GPT-3.5-turbo-0301, one of the largest available models at the time it was conducted, and the class of model currently used in the open access version of ChatGPT. The researchers also utilized the 2021 NCCN guidelines, as GPT-3.5-turbo-0301 was developed using data through September 2021. Although results may vary with different LLMs and clinical guidelines, the researchers emphasize that many LLMs share similar construction and limitations.
Exploring Future Possibilities
To further understand the capabilities and limitations of both patients and doctors in distinguishing between medical advice from a doctor and a language model, the researchers are actively investigating this aspect. They are also working on obtaining more detailed clinical cases from ChatGPT to assess its clinical knowledge more comprehensively.
It is essential to recognize that while AI technologies have the potential to revolutionize healthcare, they should augment the expertise of healthcare professionals, rather than replace them. Collaborative decision-making, where AI tools are used as aids in conjunction with clinical expertise, can help improve patient outcomes and promote responsible AI integration into healthcare systems.
Conclusion
As patients increasingly turn to the Internet and AI chatbots for medical information, it is crucial to understand the limitations of such technologies. The study conducted by researchers at Brigham and Women’s Hospital highlights the need for caution when relying solely on AI language models like ChatGPT for cancer treatment recommendations. While ChatGPT can provide valuable insights, they should be used in conjunction with professional medical advice.
By fostering collaboration between patients and doctors and acknowledging the nuances of individual patient cases, healthcare systems can harness the potential of AI technologies effectively. Continuous research and evaluation of AI tools will help ensure their responsible incorporation into healthcare delivery, benefiting patients and healthcare professionals alike.
Summary:
A study by researchers at Brigham and Women’s Hospital evaluated the consistency of cancer treatment recommendations provided by the artificial intelligence chatbot, ChatGPT. The findings revealed that about a third of the recommendations did not align with the clinical guidelines from the National Comprehensive Cancer Network (NCCN). While patients can benefit from the Internet as a self-education tool, it is essential to recognize the limitations of relying solely on AI technology for medical advice.
Collaboration between patients and doctors is crucial in leveraging the potential of AI tools effectively. By working in conjunction with healthcare professionals, AI technologies can aid in decision-making and improve patient outcomes. However, it is important to remember that AI language models like ChatGPT are not a substitute for medical expertise.
The study emphasized the complexity of individual patient cases and the need for nuanced decision-making. ChatGPT’s responses, while convincing, may lack the ability to capture the subtleties involved in clinical decision-making. Therefore, patients should always consult with their doctors and not solely rely on online resources.
Moving forward, the researchers are exploring ways to improve the understanding and distinction between medical advice from doctors and language models like ChatGPT. They are also working on enhancing ChatGPT’s clinical knowledge by incorporating detailed clinical cases.
As AI tools continue to advance, it is important to conduct rigorous research and evaluation to ensure their responsible incorporation into healthcare systems. This will pave the way for a collaborative and effective approach to patient care, where AI technologies augment the expertise of healthcare professionals.
—————————————————-
| Article | Link |
|---|---|
| UK Artful Impressions | Premiere Etsy Store |
| Sponsored Content | View |
| 90’s Rock Band Review | View |
| Ted Lasso’s MacBook Guide | View |
| Nature’s Secret to More Energy | View |
| Ancient Recipe for Weight Loss | View |
| MacBook Air i3 vs i5 | View |
| You Need a VPN in 2023 – Liberty Shield | View |
For many patients, the Internet is a powerful self-education tool on medical topics. With ChatGPT now within the reach of patients, researchers at Brigham and Women’s Hospital, a founding member of the Mass General Brigham health care system, evaluated the consistency with which the artificial intelligence chatbot provides cancer treatment recommendations that align with clinical guidelines. guidelines from the National Comprehensive Cancer Network (NCCN). Their findings, published in JAMA Oncologyshow that in about a third of the cases, ChatGPT 3.5 provided an inappropriate (“non-matching”) recommendation, highlighting the need to be aware of the limitations of the technology.
“Patients should feel empowered to learn about their medical conditions, but they should always talk to a doctor, and online resources should not be viewed in isolation,” said corresponding author Danielle Bitterman, MD, of the Department of Radiation Oncology and the General Mass Brigham’s Artificial Intelligence in Medicine (AIM) Program. “ChatGPT responses can sound very human-like and can be quite convincing. But, when it comes to clinical decision-making, there are many subtleties to each patient’s unique situation. A correct answer can go a long way.” nuances, and not necessarily something ChatGPT or another large language model can provide.”
The emergence of artificial intelligence tools in healthcare has been groundbreaking and has the potential to positively reshape the continuity of care. Mass General Brigham, as one of the nation’s largest integrated academic health systems and innovation companies, is leading the way in conducting rigorous research on new and emerging technologies to inform the responsible incorporation of AI into delivery. care, workforce support and administrative processes.
Although medical decision-making can be influenced by many factors, Bitterman and his colleagues chose to assess the extent to which the ChatGPT recommendations aligned with the NCCN guidelines, which are used by physicians at institutions across the country. They focused on the three most common cancers (breast, prostate, and lung cancer) and asked ChatGPT to provide a treatment approach for each cancer based on disease severity. In all, the researchers included 26 unique diagnostic descriptions and used four slightly different prompts to ask ChatGPT to provide a treatment approach, generating a total of 104 prompts.
Nearly all responses (98 percent) included at least one treatment approach that matched the NCCN guidelines. However, the researchers found that 34 percent of these responses also included one or more non-matching recommendations, which were sometimes difficult to spot in the midst of strong targeting. An inconsistent treatment recommendation was defined as one that was only partially correct; for example, for locally advanced breast cancer, a recommendation for surgery alone, with no mention of another therapy modality. Notably, full agreement on scoring only occurred in 62 percent of cases, underscoring both the complexity of the NCCN guidelines and the degree to which ChatGPT results could be vague or difficult to interpret.
In 12.5 percent of cases, ChatGPT produced “hallucinations” or a treatment recommendation completely missing from the NCCN guidelines. These included recommendations for novel therapies or curative therapies for non-curable cancers. The authors emphasized that this form of misinformation can incorrectly set patients’ expectations about treatment and potentially affect the doctor-patient relationship.
Going forward, the researchers are exploring how well both patients and doctors can distinguish between medical advice written by a doctor and a large language model (LLM) like ChatGPT. They are also asking ChatGPT for more detailed clinical cases to further assess their clinical knowledge.
The authors used GPT-3.5-turbo-0301, one of the largest models available at the time they conducted the study and the class of model currently used in the open access version of ChatGPT (a newer version, GPT -4, it is only available with the paid subscription). They also used the 2021 NCCN guidelines, because GPT-3.5-turbo-0301 was developed using data through September 2021. While results may vary if other LLMs and/or clinical guidelines are used, the researchers emphasize that many LLMs they are similar in the way they are built and the limitations they have.
“It is an open research question to what extent LLMs provide consistent logical answers since ‘hallucinations’ are often observed,” said first author Shan Chen, MS, of the AIM Program. “Users are likely to seek answers from LLMs to learn about health-related topics, similar to how Google searches have been used. At the same time, we need to raise awareness that LLMs are not the equivalent of professionals. trained physicians.”
—————————————————-