Medical education in Asia may be heading into a future where institutions not only train students, but also train AI models.
In the HIMSS24 APAC During the session ‘Implementing AI in Healthcare’, Dr. Hyung-Chul Lee, deputy CIO of Seoul National University Hospital (SNUH), how their hospital develops, validates and implements AI technologies.
“For more than 100 years, our hospital has been training medical students and residents. But now we also need to train LLMs (large language models).”
When LLMs finally made their way into healthcare recently, SNUH researchers quickly validated their potential application. Although LLMs scored highly in one of the world’s toughest medical trials and outperformed existing predictive AI models in use cases such as predicting hospital mortality and real-time admissions, their capabilities remain limited.
“Our research shows that no LLMs have yet scored higher than 60 (on the Korean medical licensing exam), especially in the field of medical law.”
“Also, these LLMs have limited capabilities (in performing) multimodal tasks. For example, if I uploaded images related to ventricular tachycardia – a condition requiring emergency resuscitation – from VitalDB (a multi-parameter vital signs database), even the ( latest) ChatGPT-4o responds (incorrectly) that it is a normal ECG and no CPR is required.”
Dr. Recognizing the need for LLMs to be fed with more diverse data, Lee and his team built a vector database and query refinement model. “We are currently working on vector embedding all the data in our hospital and loading it into the VectorDB.”
This initiative led to the development of SNUH’s internal LLM platform to securely develop AI models for their physicians. The LLM service, Dr. Lee said, required 40 H100 GPUs and six petabytes of storage.
“VectorDB will be our textbook and refining it will be (in) our curriculum.”
“I look forward to seeing AI models train in our hospital and (how they) will change our practice and improve patient outcomes.”