AI used by doctors to generate medical notes is missing vital information conveyed by patients’ gestures and facial expressions, a study warns.
About four in ten GPs now use AI writers, which convert speech to text and generate consultation summaries.
Ambient voice technology allows doctors to focus more on the patient and is designed to save time by reducing paperwork.
But researchers say its use could lead to patients withholding information they don’t want AI to hear, and could result in “dekilling” doctors who will have fewer opportunities to maintain their clinical reasoning skills.
Furthermore, AI summaries tend to prioritize clinical information, which can come at the expense of patients’ stories and experiences with their disease, they add.
Academics at the University of Edinburgh issued their warning after reviewing 27 articles published in the scientific and medical literature that examined the adoption of AI writers and the implications of their rollout.
NHS England has told hospitals to ‘deploy ambient voice technology (AVT) at pace’, but insists doctors should review the notes it takes and ask patients for permission before using them at appointments.
The new review highlights that these programs only capture spoken words and may miss gestures, facial expressions and the tone in which patients or doctors speak.
Ambient voice technology allows doctors to focus more on the patient and is designed to save time by reducing paperwork.
Researchers described the tools as having a “preference for written language.”
“They convert spoken consultations into text and then process the interaction as a textual record,” she added.
‘Environmental writers cannot capture the potentially vital information that a clinician can gather from gestures, facial expressions, movements and emotional affects.’
The experts say patients can ‘adapt their behaviour’ towards AI and that a ‘lack of trust’ could lead to less disclosure of sensitive information, such as substance abuse, domestic violence or mental health issues.
Dr. Lucas Seuren, research fellow at the University of Edinburgh’s Center for Biomedicine, Self and Society, said: ‘Many doctors are excited about ambient AI writers as they promise to reduce paperwork.
‘But there is little thought given to patients’ experiences and there is a real risk that patients’ stories will be lost.
‘This could further disadvantage people who already face marginalization in health and social care.’
The findings, published in BMJ Digital Health and AI, highlight that the technology ‘may require careful adaptation to clinical and healthcare system contexts’, and that ‘its implementation could reshape the way health and care work is organized and delivered’.
Researchers say the “enthusiasm surrounding ambient scribes has led to widespread adoption.”
However, most studies have focused on the benefits for productivity and technological performance, while “the socio-technical dimensions of adoption have been neglected,” researchers said.
‘Ambient scribes are complex interventions with the potential to reshape clinical work, documentation practices and the organization of care.
“Before pursuing large-scale dissemination, it is important to develop a robust evidence base on the consequences of early deployment and develop a comprehensive understanding of how care with environmental writers can and should be designed to support patients, clinicians and healthcare systems,” she added.
Some doctors report not recognizing their own AI-produced notes and finding it harder to remember patients the next time they see them.
Dr. Sarah Townley, deputy medical director at Medical Protection, which provides medical compensation to doctors, said: ‘Doctors often tell us about the burden of excessive administrative tasks and their concerns about the impact on patient care.
‘We welcome AI authoring tools that help minimize this part of their work, but recognize the important risks highlighted by this assessment.
‘Gestures, facial expressions and tone of speech are important features when assessing patients and documenting these details can also help doctors with recall when patients return. ‘The inability to capture this illustrates the limitations of this technology and the importance of a clear framework for practitioners to use AI safely.
‘Our advice to doctors is that they are responsible for ensuring that medical notes produced by transcription software are clear and legible, accurately reflect the consultation and do not omit potentially relevant information.
‘When using AI scribe tools – especially those that require sharing personal data with a third party – informed consent from patients should be obtained before use and this should be documented.
‘Doctors should also receive appropriate training on AI writers so that they understand their limitations and data protection implications and can reassure patients about how their personal information is used and stored.’
