“Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models” (2023), by Tiffany H. Kung et al. Medicine research paper. Original abstract below, unchanged. A free PDF is available.

Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models

Tiffany H. Kung, Morgan Cheatham, ChatGPT et al. · February 9, 2023 · Medicine

Abstract

We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.

Free PDF DOI: 10.1371/journal.pdig.0000198