by Sometimes, in our imagination, technology takes on a detached, almost cold connotation, influenced by our almost reverential approach to science. In fact, many of the practical applications of those technologies have a concrete effect on the daily and very personal aspects of our social life.
One example concerns the most innovative technologies and applications in the wellness sector, one of the most flourishing research lines at the Politecnico di Milano.
There is a very widespread disorder that people know very little about. It’s called emotional dysregulation. It is a neurodevelopmental disorder (NDD), a framework encompassing a group of conditions characterised by severe deficits in the cognitive, emotional and motor spheres, which produce communication impairments and affect the person’s social life. It is estimated that as much as 10% of the population suffers from this condition at various levels.
Emotional dysregulation is split into several stages of seriousness: from difficulties recognising the emotional state of others, to impediments in the production of their own, to a total inability to identify and recognise them in themselves, a state that is known scientifically as alexithymia.
We interviewed Fabio Catania, 27, a PhD student in Information Technology who is writing a thesis on the use of conversational technologies and affective computing to support neurodevelopmental disorders.
Hello Fabio, tell us a little bit about Emoty.
Emoty is a research project that uses technology with a view to alleviating the difficulties of some people in recognising and expressing emotions.
It is not a virtual assistant supporting everyday life; it is more of a “coach” that helps to develop better emotional control and self-awareness, so as to improve communication skills and consequently quality of life.
The project has been developed in close collaboration with psychologists, linguists, therapists, neurologists, caregivers and people with neurodevelopmental disorders.
What technologies are behind Emoty?
Emoty is based on conversational technologies, one of the research topics explored by the I3lab laboratory, where I am conducting my thesis. Conversational technologies are those we use every day when we interact with the well-known voice assistants.
Emoty is basically a voice that dialogues in Italian on screen, conversing with users in a natural language and entertaining them with chats and educational games.
Its goal is to stimulate conversation. There are many people who do not feel comfortable interacting with others. For them, communicating with a computer is an advantage as it involves a single channel of interaction, eliminating nonverbal communication, which can be difficult and distracting.
How does it work?
Emoty offers a variety of games, so as to engage users in a fun and interactive manner.
For example, one of Emoty’s activities involves uttering a phrase spoken with a certain intonation and then asking the user: “What emotion is being expressed?”.
Alternatively, it displays a phrase on the screen and then asks the user to repeat it, expressing a certain emotion by modulating their tone of voice.
At that point, drawing on artificial intelligence, Emoty uses machine learning and deep learning techniques to process the audio track, identify which emotion was actually transmitted by the user’s tone of voice, and give feedback.
What have you achieved so far?
Experiments have been carried out on children to verify their perception of the application and its usability.
The results were twofold. Sometimes they appreciated the human characteristics of Emoty, interacting with the machine as they would interact with a person. At other times they recognised the characteristics of a machine, endowing the computer with the authority and infallibility that only an electronic computer can have.
What does the data collected reveal?
It is still too early to tell as data collection is still in progress. I can tell you, however, that we have noticed an improvement in the children’s performance as they played with Emoty. But for the final results you will have to wait for my thesis, which I expect to publish in September.
What did you study at university?
I have always been interested in IT. Therefore, after attending a high school specialising in science studies, I graduated in Engineering of Computing Systems from the Politecnico di Milano. I then continued my studies with a Laurea Magistrale (equivalent to a Master of Science), which was the natural continuation of my degree.
What made you study the interaction between humans and computers?
During my Master, I did an Erasmus in Las Palmas, Gran Canaria. The experience left its mark as, when I returned to Italy, it made me think about what I really wanted. I no longer wanted to do something that was exclusively machine-related, but something that could be useful to people.
At the Politecnico, did you find room for these aspirations?
At that time there was hardly any literature on the use of conversational technologies in the treatment of neurodevelopmental disorders. This was an essentially a new field of research.
So I approached the Politecnico’s I3lab (Innovative, Interactive Interfaces Laboratory). This laboratory focuses on the application of innovative interactive technologies and applications in a number of challenging areas, such as wellness and education. For years, I3lab’s research has primarily targeted children with special needs, to help them develop their social, emotional and intellectual skills.
How did the idea of your PhD come about?
Let me tell you something that happened that still makes me feel emotional. I was in the process of completing my thesis for my Master, and was conducting experiments in a therapeutic centre. A girl with Down syndrome was helping me test Emoty by producing phrases with a particular emotion. At a certain point she had to repeat a sentence while expressing anger, and I have to say she was managing quite well. But Emoty went phut. After several unsuccessful attempts, she looked at me and said: “I’ll let it pass this time, but next time I’ll get really angry”.
I felt downhearted by that result and by her reaction, but the therapist took me to one side and congratulated me on my excellent work. She told me that that simple sentence contained three important results: the girl knew she was acting, she understood what anger was, and she realised that emotions can be controlled.
I imagine the satisfaction you must have felt…
At that moment an entire world opened up to me: I had seen the concrete results my tool could achieve. My goal was clear: I wanted to help people. I wanted to use a technology that we are already familiar with in a field where it had never been applied, the treatment of NDDs.
I spoke about this with Professor Franca Garzotto, the director of I3lab, who has always supported me and endorsed this project.
Have you already thought about what you will do after your PhD?
I don’t really know yet; at the moment I’m concentrating on my thesis. I would certainly like to continue developing the Emoty project. But it would take a lot more resources, because at present I’m basically working on it on my own. My dream is for it to become a project involving many different people. A project for everyone, in line with its underlying spirit.
I know you are still looking for voices for Emoty…
That’s right. The “Emozionalmente” platform I developed is a key part of my project. By acquiring volunteers’ voice recordings, it helps Emoty recognise emotions more accurately, thereby improving the tool continuously. So far we have collected 10,000 voices, but we need as many as possible. So I’d like to appeal to those who are reading this: on the Emozionalmente website you can “donate” your voice in a totally anonymous way, or even just listen to those of others and try to understand the emotion transmitted. Thank you very much.