Implementing AI on a national scale, especially in critical sectors like healthcare, is no easy feat.
In the HIMSS24 APAC During the session ‘Implementing AI-Powered Triage for a National Virtual and Digital Front Door’, Dr. Nirvana Luckraj, Chief Medical Officer of Healthdirect, described how they selected, tested and deployed an AI-powered clinical decision support system to expand the national helpline. services.
Healthdirect offers a range of free 24/7 helpline services, including nurse triage and GP telecare. It has special lines for pregnant individuals, the elderly and, most recently, those dealing with mental health issues. In addition, Healthdirect offers information about medicines and curated health content on their website and offers online bookings and referrals, which integrate directly with national care pathways, including My Health Record.
THE PROBLEM
Acquiring a CDSS is part of the actions Healthdirect took when it recently migrated its technology stack to its own operating platform.
It required the tool to consider multiple symptoms from the start. “We needed a CDSS that could tolerate and analyze multiple symptoms. We also wanted it to reflect true consultation, taking into account risk factors such as predisposing factors, chronic history and travel history.”
“We also wanted something next-generation that could provide more comprehensive and accurate triage,” Dr. Luckraj added.
Previously, the service used a CDSS based on a decision tree type algorithm that required “extensive maintenance” and could “only tolerate one symptom entry.”
PROPOSAL
The organization then went to the market to find a provider of an AI-based CDSS. Initially, five global suppliers were shortlisted and their offerings were subjected to clinical testing.
To test the applicability of the AI system in Australia’s public health system, Healthdirect collaborated with AI experts from the Australian Institute of Health Innovation at Macquarie University. They adopted a testing methodology from a recent study that used case vignettes to check the accuracy of a CDSS.
“We ran these vignettes through the CDSS and found that performance was consistent with the recently published study. We also ran the same vignette through our old CDSS to compare and found that the old and new CDSS had similar results, but that the new better was accurate,” Dr. Luckraj noted.
Ultimately, Healthdirect chose the AI CDSS from Polish company Infermedica, which runs on a probabilistic Bayesian inference engine that takes into account symptoms, risk factors and diseases, an updated medical knowledge base and an API that connects to its CRM.
MEETING THE CHALLENGE
Immediately after launching the new CDSS on the helpline in March 2023, the implementation team encountered resistance from staff. “We underestimated the change management required to get healthcare professionals using new technology to perform old tasks,” admitted Dr. Luckraj.
From entering one symptom into the CDSS, their nurses now have to enter multiple patient symptoms. “The nurses were initially frustrated with the tool because it seemed to ask seemingly unrelated questions to the patient’s presentation. They also had to take their history to determine individual risk factors and the questions also seemed more advanced than the previous ones.”
In dealing with the nurses’ initial frustration, the team took their feedback and helped them adapt to the new tool. One of the key areas for improvement in this area, according to Dr. Luckraj, was working with their supplier on a feedback loop mechanism where nurses do not have to agree with every CDSS output.
“It is a support system, not a diagnostic system. So the nurses can upgrade or downgrade the triage outcome, giving a reason for their change of setting. We also looked at those flows where the nurses disagree with the CDSS output to continuously improve the tool.”
RESULTS
One year after improving the CDSS, the implementation team noted that nurses change the triage output less and agree more with the CDSS results, and they are more satisfied with the new user interface. The tool’s accuracy is continually improving, Dr. Luckraj noted, as nurses also continue to provide feedback. On the consumer side, no serious clinical events attributable to the CDSS have been reported; they are now more satisfied with the service, with satisfaction scores reaching their highest in three years.
In December last year, the same AI-powered CDSS was introduced to Healthdirect’s consumer-facing online symptom checker, making it a “digital front door to countless alternative care options, including direct access to urgent care services and virtual emergency departments.”
“Going forward, we also want to provide consumers with tailored information specific to their symptom input to deliver an even better consumer experience,” Dr. Luckraj shared it.
Since incorporating AI into their helpline, Healthdirect claims to divert half of emergency calls to less acute services. From July 2023 to June 2024, it connected 44,000 calls to a virtual emergency department. It also booked almost 5,500 people online for GP-led urgent care services. Meanwhile, almost 350,000 people have been given advice on how to manage their symptoms safely at home. In total, Healthdirect nurses have helped book more than 370,000 GP appointments.