While AI can offer tremendous potential to improve patient outcomes and streamline medical processes, system failures can pose risks to ethics, safety, and data accuracy.
Dr. Gao Yujia, assistant group CTO of National University Health System, urged caution in implementing AI in healthcare during the keynote session ‘Unveiling the Dark Side of AI’ on HIMSS24 APAC.
“While AI has undoubtedly improved patient care, it is critical to recognize the potential pitfalls,” he emphasized.
One of the main concerns is the risk of bias in AI algorithms. Dr. Gao cited an incident during COVID-19 where an AI algorithm trained on biased data people of color who had the same conditions as other patients were disproportionately lower priority.
It is crucial, Dr. emphasizes. Gao, that AI systems are trained on diverse and representative data sets.
“Similar to a ‘garbage in, garbage out’ scenario, AI can only be as accurate as the data fed to it. Some AI algorithms do not perform as expected because they are not trained on relevant data or are over-adapted to specific data sets.”
In addition to the quality of the data, Dr. Gao also raises concerns about the ethical implications of AI in healthcare.
“We use AI to help us organize our thoughts, but that same intention can be used maliciously. For example, AI-generated deepfakes have been used to impersonate healthcare workers for ransomware attacks. Fraudulent medical research is also a possibility because of how compelling AI-generated content can be.”
When asked if all is doom and gloom for AI in healthcare, Dr. Gao disagreed.
“A more immersive and immediate digital world is inevitable, but what protects us (from AI risks) is maintaining a balance between human action and the usefulness of AI.”
According to Dr. Gao, ethical frameworks, transparency and human oversight are key factors in harnessing the power of AI and mitigating its potential harms.
He advised healthcare professionals to neither fear AI nor be fearless of AI.
“We need to take a step back to understand, appreciate, understand and plan how we can integrate AI into healthcare in our society.”