Papers

Topics

Authors

Recent

View all

Assistant

AI Research Assistant

Well-researched responses based on relevant abstracts and paper content.

Custom Instructions Pro

Preferences or requirements that you'd like Emergent Mind to consider when generating responses.

Gemini 2.5 Flash

Gemini 2.5 Flash 175 tok/s

Gemini 2.5 Pro 54 tok/s Pro

GPT-5 Medium 27 tok/s Pro

GPT-5 High 23 tok/s Pro

GPT-4o 96 tok/s Pro

Kimi K2 196 tok/s Pro

GPT OSS 120B 464 tok/s Pro

Claude Sonnet 4.5 35 tok/s Pro

2000 character limit reached

Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicine (2401.08396v4)

Published 16 Jan 2024 in cs.CV, cs.AI, and cs.CL

Abstract: Recent studies indicate that Generative Pre-trained Transformer 4 with Vision (GPT-4V) outperforms human physicians in medical challenge tasks. However, these evaluations primarily focused on the accuracy of multi-choice questions alone. Our study extends the current scope by conducting a comprehensive analysis of GPT-4V's rationales of image comprehension, recall of medical knowledge, and step-by-step multimodal reasoning when solving New England Journal of Medicine (NEJM) Image Challenges - an imaging quiz designed to test the knowledge and diagnostic capabilities of medical professionals. Evaluation results confirmed that GPT-4V performs comparatively to human physicians regarding multi-choice accuracy (81.6% vs. 77.8%). GPT-4V also performs well in cases where physicians incorrectly answer, with over 78% accuracy. However, we discovered that GPT-4V frequently presents flawed rationales in cases where it makes the correct final choices (35.5%), most prominent in image comprehension (27.2%). Regardless of GPT-4V's high accuracy in multi-choice questions, our findings emphasize the necessity for further in-depth evaluations of its rationales before integrating such multimodal AI models into clinical workflows.

Citations (17)

View on Semantic Scholar

Summary

The paper reveals that GPT-4’s multimodal vision achieves expert-level accuracy while masking critical diagnostic limitations.
It employs comprehensive case analyses to evaluate the system’s performance in complex medical imaging tasks.
Results imply that overreliance on AI predictions without human oversight may lead to misdiagnoses and compromise patient safety.

Understanding Skin Manifestations: The Case of Gingival Melanoma

Background

Skin changes and lesions can be signs of various health issues, and sometimes they can be the symptomatic presentation of rare and serious diseases. A recent clinical case highlights the importance of early detection and treatment of gingival melanoma, an uncommon and aggressive form of malignancy originating in the oral cavity's melanocytes.

Case Presentation

In the noted case, a 61-year-old woman presented to a medical clinic with concerns about the rapidly expanding discoloration along her gums—initially noticed over a year ago. Upon examination, the discoloration was dark and concentrated in irregular patches near the base of the gums.

Diagnosis and Treatment

A diagnosis of gingival melanoma was made after clinical evaluation and led to further imaging, which thankfully revealed no lymph-node involvement or distant metastases. The patient underwent successful surgical resection of the lesion. Four months post-surgery, she showed no signs of recurrence.

Differentiating from Other Conditions

The condition shared similarities with other potential diagnoses such as amalgam tattoo, Kaposi's sarcoma, oral melanoacanthoma, and physiological pigmentation—each with its distinct characteristics. However, the rapid expansion of the discoloration and its specific appearance were indicative of gingival melanoma. Its dark pigmented lesions and aggressive growth pattern distinguished it as the leading diagnosis.

Conclusion

This case underscores the critical nature of medical vigilance and patient awareness, particularly when it comes to changes in skin appearance. For healthcare professionals, it highlights the intricacy of skin lesion diagnosis and the importance of considering a broad spectrum of potential conditions. As for patients, it reaffirms the significance of promptly seeking medical advice when noticing changes in one’s body, as early detection can be lifesaving, especially with conditions like gingival melanoma.