A new AI app detects depression from facial cues

Artificial intelligence (AI) has been gaining popularity in mental health services. From AI chatbots that provide therapy to diagnosis of illnesses, it has been considered a novel tool in the field. Now, researchers have developed the first smartphone application that uses AI to detect the onset of depression before the user even knows something is wrong.

Researchers from Dartmouth University have developed an app called MoodCapture, which uses a phone’s front camera to capture a person’s facial expressions and surroundings during regular use. It examines the images for clinical cues linked to depression, the university’s press statement explained.

For the study, the researchers involved 177 people diagnosed with major depressive disorder. The app correctly identified early symptoms of depression with 75% accuracy. According to the researchers, the technology could be publicly available within the next five years. The findings were published on the arXiv preprint database.

“This is the first time that natural ‘in-the-wild’ images have been used to predict depression,” Andrew Campbell, the paper’s corresponding author said in the statement. “There’s been a movement for digital mental-health technology to ultimately come up with a tool that can predict mood in people diagnosed with major depression in a reliable and nonintrusive way,” he added.

At a time when people are heavily dependent on their phones, they tend to use facial recognition software to unlock their phones hundreds of times a day, the researchers pointed out. The new app uses similar technology with deep learning and AI hardware to identify signs of depression and can suggest they seek help.

For the study, the application captured 125,000 images of participants over the course of 90 days. The idea is that every time a person unlocks their phone, MoodCapture analyses a sequence of images in real-time. 

The AI model finds associations between expressions and background details, which are important when finding the severity of depression, the statement explains. For instance, if a person consistently appears with a flat expression in a dimly lit room for a long period, the AI model might infer that they could be experiencing the onset of depression.

According to the researchers, MoodCapture could lead to assessment tools that would help detect depression in the moments before it gets worse.

Previous studies have also shown how AI can be used in the detection and diagnosis of health issues. For instance, a study published in the journal Nature in October 2023, highlighted an AI tool EVEscape which can predict future variants of the SARS-CoV-2 virus, which causes Covid-19.