UABDivulga - Barcelona recerca i innovació

Jocomunico is an augmentative and alternative communication (AAC) system that is unique compared to existing systems. AAC is the set of tools and strategies that replace or support natural speech. In this case, Jocomunico is a pictogram-based communicator used by people with severe speech impairments that build sentences by means of boards of pictograms. 

The system is unique in that it transforms the telegraphic language resulting from pictograms into natural language, in Catalan and Spanish. For instance, a sentence like “escuela querer yo ir” (school want I go) would be translated into “Quiero ir a la escuela” (I want to go to school). 

Communication through AAC devices is slow and, when the resulting output are telegraphic-like sentences, it reduces the quality of human interaction and can cause frustration in the users. Trying to improve these aspects and, thus, their quality of life, was the main motivation behind this project.

In the course of the PhD*, which this solution is based upon, multidisciplinary research was done in the fields of AAC, machine translation, linguistics and software accessibility in order to develop and test the system in a real-life environment with end users. After the system was ready, these tests were performed during 3 months at a day care center for people with severe cerebral palsy. Aside from cerebral palsy, the 4 participants also had ranging degrees of linguistic competence and cognitive disabilities. Nevertheless, as all of them had used pictogram-based AAC for the past 30 years, their linguistic abilities were stable. This was very important in order to ensure that the observed changes would be due to the introduction of the new system.

During tests, participants were asked to produce prepared sentences and also to participate in one on one conversations adapted to their skills. By the end of the last session, participants were able to learn new linguistic skills, such as the use of basic verb tenses, as well as slightly increase their communication rate (number of words per minute).

Furthermore, and more importantly, the language expansion system also proved a source of motivation and, based on the feedback from participants, caregivers and speech language pathologists from the center, it improved the overall communication experience, as the output was in natural language. 

Finally, it is important to highlight that the system fit well with the linguistic competence of each person and required no learning curve during tests when none of its special features (different from most communicators), aside from language expansion, like gender, number, verb tense or sentence type modifiers were used, showing that it can be easily introduced to new users familiar with pictogram-based communication. 

Results shown in the research were later backed by the fact that, since its release, Jocomunico has been adopted by more than 1000 users throughout Spain.

*The thesis was directed by Jordi Herrera Joancomartí.