Sub-threshold depression recognition from both the face and the voice

New research published in Springer Nature https://www.nature.com/articles/s41598-025-15874-0 by two researchers from Waseda University on Japanese students showed that sub-threshold depression can be detected using facial muscle action detection. The so-called Action Units were automatically detected with the open-source Open Face 2.0 tool.

What’s interesting about this paper is that it looks at objective measures of depression from the face, rather than from the voice. Over the years, there have been many studies that have shown good results for detecting the presence or severity of depression from the voice. There have been much fewer published studies for depression recognition from the face.

This is a shame, as the two can be highly complementary. There are many scenarios where people are simply not speaking (e.g. while driving a car, or in a quiet train coach) or where there is too much background noise for voice data to be a viable modality. Purely visual approaches would still work in such cases. More importantly, in BLUESKEYE AI’s own studies we have time and again found that best results are obtained by combining the face and the voice data. And why wouldn’t you? If you’re collecting voice data, it really isn’t a much bigger step to add face data. They’re both special category data, and from a privacy perspective I would argue that the content of what you say is actually more sensitive than what you look like.

If you’re a researcher, wouldn’t it be great if there was a tool that you could use to robustly, privately, and securely capture face and voice data and automatically get indicators of depression from both channels? This is exactly what you would get from BLUESKEYE AI’s B-Healthy Platform.

BLUESKEYE AI's Action Unit detection as included in our B-Healthy Platform clinical trial support tool is roughly twice as accurate as Open Face 2.2, which should result in even stronger evidence in the context of depression research. We have already shown the efficacy of this in perinatal depression, where we achieved a sensitivity of 83.5% and precision of 97.2% to distinguish between concern and no concern on a cohort of a thousand pregnant women, which is now being validated with

We think that anyone who's interested in adding objective measures of depression as a secondary endpoint to their clinical trial should give B-Healthy a try! It doesn’t matter if you’re thinking of a small-scale study or a large one. And if you don’t know exactly how to start collecting data with a mobile app and clinical grade secure backend, don’t worry, we have the excellence in digital health to get you started.

Get in touch today to arrange a call!