Braille Interpreter

IDEA

• The application is expected to capture the image of a Braille sheet and convert it to text. Braille is a method that is used by blind people to read by the sense of touch. This application is intended to make the process of reading or learning Braille easier for those who do not understand the language. There are three grades in Braille, wherein Grade 1 uses the letters in the alphabet set and numbers. The other two grades are forms of shorthand. The application is aimed at reading and interpreting Grade 1 Braille. It can be enhanced later to interpret Grades 2 and 3.

• Novelty of Idea
  • In Braille, there are 6 dots arranged in a 3 by 2 matrix, designated to each letter/number in the alphabet. Every letter/number is identified by the pattern formed by the dot/s that is/are raised, among the 6 dots assigned to it. In an image, the raised dots will be differentiated from the plain dots in order to reach the goal. The pattern of the larger dots will be used to identify the letters/numbers.
  • The application is unique in itself as it will act as a translator that converts Braille to text.

• Relevance and Applications

  Braille is a language that is common among the visually impaired, but since it consists of many patterns for identifying letters and numbers, it becomes difficult to understand.

Currently, a variety of text to Braille applications is available, but there is no application which does the reverse. There are desktop applications which convert Braille dots which are provided in soft copy format into text, but they are not images of Braille sheets. Many papers have been written on Braille Recognition but there is no implementation of the same, esp. on a mobile phone.

The new application is aimed at bridging the gap between the visually impaired and the sighted.

The application can also be used in the evaluation of Braille assignments, submitted by students on Braille sheets, by examiners who do not understand Braille.

Hence the sighted need not spend much time in learning and understanding Braille. This can eliminate the need for training the staff in Grade 1 Braille in Braille schools, thus saving time and cost, which would otherwise have to be incurred to do the same.

SOLUTION 

• Technology Layer/Solution
  • Platform : Android 2.3.3
  • Language : Java
  • SDK : Eclipse 3.5.0
  • Architecture: The sequence diagram is given below:

R&D

• R&D Elements in the solution
  • A sample Braille sheet image has to be examined so as to understand the difference in diameter/colour of the raised and plain dots.
  • Image processing algorithms need to be devised or explored in detail, to parse the Braille image.

• Improvements on existing solution/ a new solution
  • Current research papers on Braille recognition suggest that the dots can be differentiated using the shadows they form in the image, when light falls on the sheet. But the formation of shadows depends on the angle at which the light falls on the sheet.
  • Some papers suggest that weak and strong edges should be detected and the weak edges are output only if they're connected to the strong edges.
  • One solution could be to read the image from left to right and from top to bottom.
    • Based on the diameters of the circles, the raised dots can be differentiated from the plain dots.
    • Or, based on the colour of the raised dots and those of the plain dots, they can be differentiated.

    Once the dots have been differentiated, the pattern of the raised dots in each matrix can be used to identify the character they represent.

INTERMEDIATE STATUS

The first image is that of a braille sheet, which contains letters from a-u,v,x,y,z,w, and then a comma, a semi colon, opening and closing quotes, and opening and closing brackets. The last row is that of numbers from 1 to 8.

The second image is the image of the braille sheet where the colours have been converted to black(for dots) and white(for the rest).

The third sheet is the image in which the blue lines are the sections(rows) into which the image has been divided, the green lines at the top row indicate the top and bottom of the third character or block('c'), and the red lines there indicate its left and right. Similarly, the magenta lines in the last row indicate the top and bottom of the second last character(number indicator,which indicates that the next character is a number), and the red lines there indicate its left and right. So the last set of dots gives the output as 8.

So now, using THIS sheet, it gives proper results, it can read letters, words and numbers as well.

SCREEN SHOTS

The full transcript is :

The full braille sheet is :

A small part of this sheet has been captured and interpreted.