OpenAI’s ChatGPT was tested in a real-world medical situation by feeding it anonymized History of Present Illness notes for 35 to 40 patients. The results were fascinating, but also fairly disturbing. For roughly half of the patients, ChatGPT suggested six possible diagnoses, and the “right” diagnosis was among the six that ChatGPT suggested. However, ChatGPT’s worst performance happened with a 21-year-old female patient who came into the ER with right lower quadrant abdominal pain, where it missed a somewhat rare diagnosis.
