You may have heard about telepsychiatry, where patients connect with mental health professionals via video conferencing, but what about simpsychiatry?
SimSensei is a software program featuring a virtual human that interacts with real life patients to assess their depression level. The program was developed by computer scientists at the University of Southern California and uses the XBOX Kinnect technology to measure patients’ facial expressions, body posture, acoustic features, linguistic patterns, and other movements as they respond to a series of questions.
SimSensei can process the patients’ movements and respond in real time. In a video that demonstrates how the program works, the SimSensei recognizes that a patient averted his eyes while answering a question about the last time he felt happy. SimSensei addresses his hesitation with her next question and probes deeper. In another example, when a patient responds positively to the question, SimSensei moves onto another topic.
The quality of the simulation is pretty impressive. SimSensei can mimic a patient’s smile in order to build rapport and will lean forward to show interest in an answer. The voice doesn’t sound as robotic as Siri, but looking at the animation, you can tell you are not speaking to an actual person.
Still, Stefan Scherer, post-doctoral researcher on the project, says the program’s use of body language and speech patterns to help identify signs of depression is an improvement over the current method for depression screening.
“Presently broad screening is done by using only a checklist of yes/no questions or point scales, but all the non-verbal behaviour is not taken into account,” Scherer said. “This is where we would like to put our technology to work.”
I think it would be interesting if this technology could be incorporated into telepsychiarty. Similar to what is shown in the video, mental health professionals could see data (i.e., gaze attention, activity level, etc.) and use the information to evaluate the patients’ responses. It seems to me like a good way to get the benefits of the facial recognition software, without having the patients feel like they are talking to a robot.
