NIH-funded smartphone app uses AI to detect depression from facial cues

Depression can live in the brain, but scientists have developed a new smartphone app to detect the disorder by looking for clues on your face.

MoodCapture uses AI to assess micro changes in a person’s face – such as their gaze, eye movements and how the person tilted their head – to determine whether they were depressed.

The app, which was funded by the National Institute of Health, takes photos with the front-facing camera and sends an alert if it detects a trend in facial expressions by looking at the position of participants’ lips, eyes and depression lines . their face.

According to the study, MoodCapture was correct in identifying people with depression 75 percent of the time.

About eight percent of American adults are diagnosed with depression each year, which amounts to roughly 21 million Americans

More research needs to be done, but researchers say MoodCapture could be available to the public within five years.

“This is the first time that natural ‘in-the-wild’ images have been used to predict depression,” said Andrew Campbell, lead author of the study and professor of computer science at Dartmouth College.

“People use facial recognition software to unlock their phones hundreds of times a day,” he said.

‘MoodCapture uses a similar technology pipeline of facial recognition technology with deep learning and AI hardware, so there is great potential to scale this technology without any additional input or burden on the user.’

The study analyzed the participants’ facial expressions by looking at the angle of their facial features, such as the way their eyebrows folded, how they tilted their head and whether their lips were turned up or down.

Over time, the app noticed patterns specific to the user and correctly identified users who often had a flat expression (their facial features did not change) and who were in a dimly lit room for extended periods of time as depressed.

An estimated 60 percent of people diagnosed with depression do not seek help

Researchers recruited 181 participants in the US who had done just that depressive disorder according to a questionnaire they were asked to complete.

Participants were given three surveys daily to assess their mood and MoodCapture discreetly took up to five photos when participants answered a specific question, such as ‘I have been feeling down, depressed or hopeless’. to see if it could correctly identify that feeling.

“We chose this question because we believed it would best capture participants’ real emotions regarding depression,” the study said.

The images were captured randomly over 90 days with the front-facing camera, looking at specific facial expressions of 177 participants, including gaze, eye movements, lighting, how the person positioned their head and others.

The researchers collected a total of 125,335 images, but left out 15,063 photos that were too blurry, did not show a face, featured children or contained nudity.

MoodCapture looked at the dominant colors in the participant’s environment, the lighting conditions, where the photo was taken (indoors or outdoors), any background objects that could be used to measure the user’s activities, and the number of people in the image.

Such details, such as dim lighting, can provide insight into a person’s mental state.

MoodCapture can sequence the images in real time, combining facial features and background information to predict the severity of their depression.

Researchers asked a series of questions to determine if the person had depressive symptoms and correlated this with MoodCapture’s findings

MoodCapture not only determined whether the participant was experiencing depressive symptoms, but also suggested preventive measures, such as going outside or asking a friend for help.

“Telling someone that something bad is happening to them has the potential to make things worse,” said Nicholas Jacobson, co-author of the study and assistant professor of biomedical data science and psychiatry at Dartmouth’s Center for Technology and Behavioral Health.

“We think MoodCapture opens the door to assessment tools that can help detect depression in the moments before it gets worse,” says Jacobson.

Major depression affects more than eight percent of American adults each year, amounting to an estimated 21 million people, but an estimated 60 percent of people diagnosed with depression do not seek professional support, according to Healthline.

Researchers said the results of the study were promising, and while more research needs to be done, Campbell said they estimate this technology could be available in the next five years, adding: ‘We have shown that this is feasible is.

‘This demonstrates a path to a powerful tool to passively evaluate a person’s mood and use the data as a basis for therapeutic intervention.’

However, researchers recommended that MoodCapture and other similar applications should not be used alone and should instead be combined with other interventions for people with depression.

“Our goal is to capture the changes in symptoms that people with depression experience in their daily lives,” Jacobson said.

‘If we can use this to predict and understand the rapid changes in depression symptoms, we can ultimately prevent and treat them.

“The more we can be in the moment, the less profound the impact of depression will be.”