GenAI is exciting, but ‘algorithmovigilance’ is a must

The entire healthcare industry is abuzz with the buzzword that is generative AI. But how big can genAI really be when it comes to improving healthcare quality and enabling cost-efficiency? That’s the big question – and it will be explored at HIMSS24 in a session titled: GenAI has talent: can it save healthcare?

IT decision makers have most likely asked themselves versions of the panel discussion’s central question: “Is this really the hope for the future of healthcare or just digital snake oil in a killer package?”

In Orlando, three of the top thinkers in healthcare IT: Dr. Keith J. Dreyer, Chief Data Science Officer and Chief Imaging Information Officer at Mass General Brigham; Sonya Makhni, director of solutions at Mayo Clinic Platform; and dr. Yaa Kumah-Crystal, clinical director of healthcare IT in pediatric endocrinology at Vanderbilt University Medical Center – will explore those and other questions with moderator Don Woodlock, head of Healthcare Solutions at InterSystems, and discuss the promise and limitations of generative AI models in a range of clinical and operational use cases.

We asked Kumah-Crystal (who also speaks on another HIMSS24 panel focused on empowering physicians with AI) for her views on what artificial intelligence can and cannot do in healthcare.

Q: How excited are you about generative AI and its potential to improve healthcare?

A. As someone who has dedicated a significant portion of my career to the study of natural language processing and human-computer interaction, I am incredibly excited! My colleagues and family have learned to tolerate my frequent conversations on this topic. Like many others, I was amazed and surprised by the advanced capabilities of generative AI tools like ChatGPT when I first encountered them a little over a year ago. They have the potential to facilitate clinical documentation and information summarization for both healthcare providers and patients and provide new insights into clinical care, ultimately improving the quality of care delivery. This Cambrian explosion of progress in AI has been very exciting and has brought new hope to some of the wicked problems we have faced in healthcare.

Q. What do you think are its limitations, and what are some aspects of genAI that we should be skeptical or cautious about?

A. While advances in AI are exciting, we need to think carefully about the implications during this exploratory phase. We need to be cautious and still have healthy skepticism, especially when it comes to the safety of the algorithms and possible biases. This is why we follow the principles of “algorithmic surveillance,” a term coined by our Chairman of Biomedical Informatics, Peter Embito systematically monitor and prevent harmful effects.

Q. Where is AI used at Vanderbilt University Medical Center? Which use cases show the most success?

A. At VUMC we are investigating various AI tools, including environmental documentation, AI guidance for patients and message generation. We have deployed a HIPAA compliant version of OpenAI’s large language model in our Azure environment. This is available to our VUMC community as a secure application for education and research, which shows promising engagement.

We are both ambitious in our goals and cautious in our approach to ensuring safety.

Dr. Yaa Kumah-Crystal, Vanderbilt University Medical Center

Q. Are there plans to expand its use in the future? Is the healthcare system ambitious or cautious in how it rolls out AI?

A. Yes, we have plans to expand these applications in the future. Our researchers investigate how AI can facilitate decision support in safe and creative ways. We also integrate these tools into medical education to help our trainees learn how to use them effectively as they become more integrated into future work and ultimately become standards of care. We are both ambitious in our goals and cautious in our approach to ensuring safety.

Q. What do you hope attendees will learn from the HIMSS24 panel discussion?

A. We are in an exciting era of new possibilities with generative AI. There is a sense of optimism and hope about how we can improve patient care. I hope attendees will learn about the potential of these AI tools, the systematic approach to their safe and effective use, and the importance of algorithmic monitoring in preventing adverse effects.

The session, “GenAI has talent: can it save healthcare?” is scheduled for March 12 from 10:30-11:30 a.m. in room W207C at HIMSS24 in Orlando. More information and registration.