Braille Messenger: SMS Sending Mobile App for Blinds Using Braille

Abstract

The mobile phone is one of essential device for people in day to day life. Mostly they use mobiles for communication, entertainment and scheduling tasks etc. Among those tasks when considered about the communication, people use voice calls, online chatting, Short Message Service (SMS) to communicate with each other. But typing a message is not much easier for blinds or Visually Impaired (VI) people. At the beginning of the mobile era, mobiles have tactile buttons (hard keyboard). So typing texts using tactile buttons is much easier for blinds than using touch screens. But with the increases of mobile technology, the market targets the best featured mobiles with accessibility features (Screen Reading feature) like Voiceover in IOS, Narrator in Windows and Talkback in Android etc. So blinds also could to move on smart mobile phones. But at the beginning, to type texts on smart mobiles just used same QWERTY or 4X3 soft keyboards that sighted people are used to input texts. In this method blind user need to move finger on keyboard then system speak out the touched key and if user need to input that key need to double tap on that key. But when consider about blind or VI people their familiar way of reading and writing is the System of Braille which founded by Frenchman Louis Braille. So designers have introduced braille to text method to type texts. But when designing the app by targeting braille input, Multi-touch capability of the device must be considered. Even though most of mobile phones have Multi-touch capability, count of points that can be detect simultaneously is different. It can be 2, 5 or 10 etc. So if someone come up with a design with using 6 point of multi-touch features that not suitable for devices which having less number of multi-touch points than 6 and app won’t produce the expected output. As a solution for that problem if someone come up with a design with using only basic multi-touch feature (2 points), that design reduce the efficiency and usability who have mobile devices which capable with best multi-touch feature (10 points). Therefore, I come up with a solution by giving different User Interface (UI) designs by checking multi-touch capability of the device. I developed 3 different UI designs to support for mobile devices with having different multi-touch capabilities. Design A: Type a single braille character using 2 fingers & needs to tap 3 times to insert a single character. Target the devices which have only basic multi-touch capability of points of 2. Design B: Type a single braille character using 3 fingers & needs to tap 2 times to insert a single character. Target the devices which have multi-touch capability with less than 6 points but greater than 2 points. Design C: Type a single braille character using 6 fingers & by single tap can insert a single character. Target the devices which have best multi-touch capability of points of 10 or more than 6 points. Here at the first user have to register reference points one by one. Because here I design the user customizable UI which means no restriction way of putting fingers on screen. User just need to register fingers for position 1, 2,3,4,5 and 6 respectively. Then I used K-NN algorithm to detect input finger. I considered each reference points’ (x, y) coordinates as center of each class. Here I assume that user will not reposition his/her hand from the device. But with repeatedly touching users’ touch points automatically drifting from the first registered reference points and it may cause to increase error rate. So here I used K-Mean algorithm to update reference points/centers of each class with each single user tap. If user repositioning his/her hand he/she has to register reference points again since there is a greater variance between registered reference points and currently touched points. Here I using 6-bit Braille encoding method with voice and vibration feedback. Most of apps use Text-To-Speech (TTS) engine to read text. Here I included vibration rhythms to identify braille characters for blind-deaf people. But this feature available only for Grade 1 Braille system. Moreover, Braille Messenger to become more user-friendly I have used some simple patterns to run commands like adding WHITE SPACE, BACKSPACE, ENTER etc. To determine those patterns, I store the coordinates of draw pattern and then by using Mathematical algorithm I classify the command. As well as I hope to provide the most frequently using words which have more than 5 characters as predicted word. But here I just hope to provide a single word (most frequently used word) rather than presenting list of all prediction words. When I tested this app with participate of 3 blind people including one pseudo blind averagely I got the 92.3% of accuracy of detecting inserted braille characters and 95% accuracy of detecting draw pattern commands. With the time, speed of typing on design A, B & C was increased respect to number of sessions tried and with the 2 hand I got the maximum speed of typing which was 16 WPM.