One of the biggest challenges for deafblind individuals is communication, a reality that inspired a group of researchers and Biomedical Engineering students from the Mexico City Region of Tec de Monterrey to develop a groundbreaking device. Their project aims to bridge this gap by creating a tool that translates Braille to speech and speech to Braille.

José Rubén Fuentes, a researcher at the School of Engineering and Sciences, explains that the prototype they designed works much like a Braille typewriter but incorporates artificial intelligence (AI) and specialized hardware to enable real-time, two-way communication between deafblind individuals and those without this condition.

“The text typed on the machine is processed by an AI algorithm and converted into words that can be easily understood by another person—it’s then read aloud. On the other end, a microphone captures natural speech, recognizes the language, and transforms it into Braille,” Fuentes explains.

This innovation has the potential to greatly improve communication and social integration for deafblind individuals, making learning more accessible and reducing school dropout rates. Additionally, the device offers an alternative means of interaction in places like schools, hospitals, and businesses, benefiting not only the deafblind community but also older adults with sensory impairments.

According to a global study by the World Federation of the Deafblind, children with this condition are up to 23 times less likely to receive an education compared to their peers. In fact, their chances of attending school are 15% lower than those of children with other disabilities. In Mexico, deafblind individuals make up around 6% of the population with a disability—over 465,000 people.

Two-Way Communication: From Braille to Speech and Speech to Braille

The prototype for this device was inspired by the Perkins Machine, a tool used for writing in Braille. However, traditional Braille typewriters have a major limitation—they allow users to share information but not receive it. This challenge drove researchers to develop an innovative solution.

They designed a 3D-printed system that integrates a low-cost mini-computer (such as a Raspberry Pi or BeagleBone), Linux-based open-source software, and an electronic interface equipped with a microphone, speaker, vibration motors, and a redesigned keyboard layout.

Traditional Braille typewriters typically have 10 keys, which can be restrictive for Latin-based alphabets due to the lack of certain letters and symbols. To address this, the team added five additional keys for accents, umlauts, and letters like ‘ñ.’ These keys were strategically placed to maintain ergonomic comfort, allowing for smooth operation with both hands.

The system converts the typed Braille text into written language, which is then transformed into speech and played through a speaker, making it audible to others.

For receiving information, the system uses a microphone to capture spoken language. It processes the audio using a mathematical transformation (Fourier Transform) and converts it into a spectrogram—a visual representation of sound. This spectrogram is then analyzed by an AI algorithm (a convolutional neural network), which classifies and interprets the audio into text before converting it into Braille.

Once the message is converted, the deafblind user can receive it through a haptic system, meaning vibrations with specific patterns that correspond to Braille characters. These vibrations, generated by small motors, can be felt at designated points on the device. Additionally, the translated Braille text can be converted back into speech, allowing the sender to hear and confirm the message they shared.

“We removed the need for paper and complicated processes, making communication feel much more natural and effective,” says Fuentes.

Evaluating the System’s Functionality

In parallel, researchers and students worked on both the software and the physical design of the device to successfully integrate artificial intelligence. They also collected audio samples and data to build a word set and train the algorithm.

Biomedical Engineering also encompasses aspects of Biomechanics, so the team considered anthropometric measurements to ensure the device is as comfortable as possible for users. The keys and vibration motors were positioned to allow deafblind individuals to easily identify Braille language symbols.

“We conducted a series of tests to determine whether individuals who are not native Braille users could operate the tool with ease. We reached a probability of over 50% that users would not make errors,” Fuentes explains.

During the validation and algorithm adjustment process, both untrained participants—who had never used Braille communication tools—and experienced deafblind users were involved. Those proficient in Braille achieved an impressive accuracy rate of 95% to 98% when using the keyboard.

Fuentes emphasizes that for the system to work effectively, an active transmission and reception process is required—in other words, a true two-way communication system, where speaking and listening happen naturally, just like in a conversation.

Currently, the system is considered a support tool and is in development at a Technology Readiness Level (TRL) 5, meaning there is a working prototype undergoing validation phases. There is also potential for integrating generative AI to enhance communication, minimize errors, and incorporate an auto-correction system.

“The overall goal of this project is to provide a tool that can make a real difference for people with this condition. That’s why we’re using as many open-source resources as possible, with the vision of eventually making this solution freely accessible,” he adds.

Were you interested in this story? Do you want to publish it? Contact our content editor to learn more marianaleonm@tec.mx