IBM researcher Alexander Sorin (pictured) talks about the work he is doing with HERMES, the EU project that seeks to provide memory assistance to the elderly.
What kind of technology interfaces will the elderly feel comfortable with?
HERMES serves as an exploratory research project that investigates how the older generation interacts with devices and technology. Much of the consortium's work will involve gaining insight into the types of interfaces that are appropriate for this very heterogeneous population.
One kind of interface, for example, may be based on the TV and a simple remote control. These are interfaces with which everyone is familiar. Another interface might be a touch surface, such as a table onto which information can be projected and then selected like cards.
How are you planning to deal with the ‘big brother’ aspect and ensure user privacy?
The HERMES project includes a work package called Privacy Integration and Ethical Watch that deals specifically with ethical issues. The consortium’s management committee also includes an ethics advisor to ensure that all privacy issues are dealt with. All partners in the consortium studied the privacy regulations relevant to their local geographies and are well versed in the correct ways to deal with the data collected during the user study, whether forms filled out by users or data recorded by video and sound. This information is similar to the confidential data collected by any company that conducts surveys.
In the second phase of the project, when actual data is collected in the home system, the information will be password-protected and secured via standard software methods as are many home computers. Because of these privacy concerns, the information will not be stored on the Internet, nor will sensitive information be transferred via non-secure wireless technologies. This is also the reason that the information processing is not currently planned to be provided as a service over the web.
How did IBM get involved in this project?
In 2007, we attended an EU event in Cologne Germany, which included networking sessions specifically targeted at bringing together potential partners. Even before attending the event, we had been in touch with CURE, the project coordinator, and took an interest in their idea to develop a system that would offer cognitive assistance to the elderly. It was clear to them that speech technology would play a major role in the project. We decided to join forces and then gathered additional partners to develop a more detailed proposal and plan, which ultimately was selected for EU funding.
What kind of speech technology is the Haifa Lab contributing to HERMES?
Even though the HERMES home of the future collects both video and audio information, the majority of information that will be collected and processed is represented by speech. Moreover, it is easier to extract semantics from and search in speech because it can be converted to text. Recognition of speaker identity and emotions provide additional dimensions for search in spoken data. This makes our role even more crucial.
IBM researchers are responsible for all areas of speech processing in the HERMES project. Right now we’re working on speech to text transcription, speaker identification and search within speech data. Our next steps will include voice-based emotion detection. The data collected by HERMES system will be searched based on content that was spoken, on the identity of the person who participated in the conversation and said certain things - even based on emotions.
When developing a new speech system, you need to first record speech samples and manually process them in order to build baseline references. This processing enables us to train the system and to compare the actual output of the system to the reference.
Naturally, we have been facing a number of very intriguing challenges. Classical speech processing works differently on close-up microphones as opposed to PDA recording and distant microphone recordings. Additionally, elderly voices generally mean a lower recognition rate in standard speech systems. We need to adapt the system so it has high accuracy with different kinds of recordings, elderly voices, and lexicon related issues. We are already working with recordings from tens of elderly people in Spain and Austria. Our researchers are also building the software required to ensure that our system interfaces with technologies introduced by other partners in the consortium. All of these challenges provide a fantastic opportunity to advance our research.
What will you remember best about this project?
I’m really enjoying making a contribution that will enable the elderly to live more independently. I have an aging mother who is unable to benefit from new technology advances -- even finding it difficult to use a mobile phone. I’m looking forward to seeing technology become accessible to the elderly in a natural and friendly way. I truly believe this will give them more self-confidence for everyday activities, such as visiting the bank or the doctor's office. The elderly often find these errands confusing and are fearful that they’ll forget important details. With HERMES, every conversation can be saved and revisited, or shared with family members.
Aside from the contribution to the elderly, I find the close collaboration with our academic and industry partners very exciting. Working together with other organizations and seeing a different approach to problem-solving is a unique and inspiring experience.
This article first appeared on the IBM intranet and was written by Chani Sacharen.
Last updated on December 3, 2008