Avatars Enlisted to Track Patient Progress

SAN MATEO, Calif. — As the demand for health care rises, more medical facilities are turning to technology to pick up the slack. One such facility is San Mateo Medical Center, which enlisted an avatar named Molly to help gather information from patients.

Molly is a computerized “therapist” that asks physical therapy patients questions in English or Spanish to help determine their pain levels, while a video guides the patients through exercises and Microsoft Kinect 3D cameras measure their movements. Molly can modulate her tone of voice and facial expressions based on the conversation.

Under the supervision of Paul Carlisle, PT, MPT, GCS, the facility’s director of rehabilitation services, the pilot project is currently conducted on site in hopes that the avatar will eventually be available to patients remotely at their homes.

“Subjectively the patients appear engaged and intrigued by the device. Certainly, there is a buzz of interest in the gym when the avatar starts speaking. The real question is whether or not the patients will use the devices in their homes. I suspect there will need to be both a human and artificial correspondence to make this really work. Patients need to know that there is a substantive benefit before making the commitment to consistent interactions,” Carlisle said.

Remote medical care continues to increase with technological advancements. In fact, Sense.ly, the California company conducting the project, is one of more than 500 companies using health care tools from Nuance, a company that creates speech-recognition and virtual-assistant software, according to MIT Technology Review.

Using speech recognition, body recognition, augmented reality, video and medical devices, the Sense.ly platform allows patients to communicate their symptoms, medical history and recovery progress to an avatar via a phone, desktop or TV. If programmed by a doctor, the avatar can even answer simple questions from the patient. The software also collects data from the patient’s medical devices.

A report is then sent to the doctor, including any red-flag notifications that need to be taken care of immediately; charts, graphs and analytics that highlight the patient’s progress; and a transcript of the patient-avatar interaction.

As for the future of the technology, Carlisle said that the medical center staff is still developing the application with Sense.ly. “We believe there will be a place for using the system on a wider basis in the future. We’re particularly interested in how we can utilize the technology to remotely monitor patient progress for various diagnoses, including low back pain and frozen shoulder syndrome,” he said.

In response to the fear that the technology use will translate to the loss of a personal touch, Carlisle said, “There is still a need for therapists to evaluate the appropriateness and set the parameters of the program. There is no doubt in my mind that patient needs will continue to outweigh the human resources available to meet those needs, in which case we have a responsibility to explore technological solutions, but there is no substitute for the human touch.”