B.V. Bhoomaraddi College of Engineering and Technology, Hubli Department of Electronics and Communication
SIGHT-Document Reader for Impaired Vision
A Major Project by
Medha V G - (2bv14ec047) Pooja Nirani - (2bv14ec059) Prajwal Prajwal Batli - (2bv13ec (2bv13ec070) 070) Under the guidance of
Dr. Priyatam Kumar
B.V. Bhoomaraddi College of Engineering and Technology, Hubli Department of Electronics and Communication
Certificate
Vision This is to certify certify that SIGHT-Document Reader for Impaired Vision
is the bona fide work of the team members Medha V G (2BV14EC047),Pooja Nirani (2BV14EC059),Prajwal Batli (2BV13EC070) of 8th semester BE, De-
partment partment of Electronics Electronics and Communicatio Communication n Engineering. Engineering. The above team members have completed the project as per norms of B.V. Bhoomaraddi College of Engineering and Technology and the Vishweshwaraiah Technological University.
Guide Dr. Dr. Priy Priyat atam am Kuma Kumar r
Examiners 1. ..................... ....................... .. 2. ..................... ....................... ..
HOD Dr. Dr. Priy Priyat atam am Kuma Kumar r
Principal Dr.P Dr.P.G .G.T .Tew ewar arii
ACKNOWLEDGEMENT
In this segment we would like to remember all those who have been a part of our project in one way or the other. Our humble thanks to Dr.Priyatam Kumar for continously instructing us to improve the things and in helping us to get things done on or before time. Also, we thank our review committee for their judgements and advice in improving the project. We are very grateful to Dr.Priyatam Kumar, Head of the Department, Electronics and Communication Engineering, for his cooperation and moral support throughout our project. We avail this opportunity to thank the Principal Dr.P.G.Tewari and the vice chancellor Dr.Ashok Shettar , B.V.B. College of Engineeering and Technology(VTU),Hubli, for all the facilities provided to us and supporting us in all academic endeavors. Finally, we take this opportunity to express our gratitude and respect to all those who directly or indirectly helped and encouraged us during the course of the project.
2017-18
Dept. Of Electronics and Communication Engineering
Page i
Abstract
- PiTextReader allows someone with impaired vision to read text from envelopes, letters and other items. It snapshots an image of the item, converts to plain text using OCR (Optical Character Recognition) and then speaks the text using text-to-speech.The Reader is designed to be as absolutely simple to use as possible. No Internet needed, no graphical interface, only one button. Just place the item to be read onto the stand and press a button. After a few moments, the text will be read back to them.
Contents 1 Introduction
1
1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1.3 Proposed Work . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
2 Literature Survey
3
3 Conceptual Design
7
3.1 Theoretical background . . . . . . . . . . . . . . . . . . . . . . . 3.1.1
7
SIGHT-Document Reader for Impaired Vision . . . . . .
7
3.2 Tools Used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
3.2.1
RASPBERRY PI 3 . . . . . . . . . . . . . . . . . . . . . .
7
3.2.2
Raspberry Pi Camera . . . . . . . . . . . . . . . . . . . .
8
3.2.3
TESSERACT OCR . . . . . . . . . . . . . . . . . . . . .
9
3.2.4
FLITE TEXT TO SPEECH . . . . . . . . . . . . . . . .
9
4 Methodology Planned
10
4.1 Steps Followed . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
4.1.1
Operating System Setup and Configuration . . . . . . . .
10
4.1.2
Application Software Installation . . . . . . . . . . . . . .
11
4.1.3
Hardware Construction and Raspberry Pi Connections
.
11
4.1.4
Testing and Operation . . . . . . . . . . . . . . . . . . . .
12
i
5 Detailed Design
13
5.1 Block diagram of SIGHT:Document Reader for Impaired Vision .
13
5.2 Circuit of the SIGHT-Document Reader for Impaired Vision
. .
14
Working of the circuit . . . . . . . . . . . . . . . . . . . . . . . .
16
5.3
6 Results
17
6.1 Expected results . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
6.2 SIGHT-Document Reader for Impaired Vision . . . . . . . . . . .
18
6.3 Budget Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
7 Future scope and conclusion
20
7.1 Future Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
7.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
8 References
2017-18
Dept. Of Electronics and Communication Engineering
22
Page ii
List of Figures 2.1 Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
2.2 Refreshable Braille Display . . . . . . . . . . . . . . . . . . . . .
6
3.1 RASPBERRY PI 3
. . . . . . . . . . . . . . . . . . . . . . . . .
8
3.2 Raspberry Pi Camera . . . . . . . . . . . . . . . . . . . . . . . .
8
5.1 Block diagram of IOT based surveillance camera . . . . . . . . .
14
5.2 Circuit diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
6.1 camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
6.2 Budget for components . . . . . . . . . . . . . . . . . . . . . . . .
19
iii
Chapter 1
Introduction 1.1
Problem Statement To design a portable device based on Raspberry Pi and Optical Character
Recognition(OCR) which captures a text and object content, converts it to speech and speaks out the text through a headphone or speaker connected to its audio jack.
1.2
Motivation Assistive technology helps persons who are blind or visually impaired
(with and without additional disabilities) increase their access to the general curriculum, improve academic performance, and can also provide a means for job and career access. It is important to thoughtfully consider what devices, tools and technologies will be appropriate to meet the person’s individual and unique learning and visual needs. When considering AT devices for students, keep in mind that AT devices should not give students an unfair advantage, but instead, should provide them with the independence to compete effectively with peers. The majority of persons who are blind or visually impaired will need some form of assistive technology in order access print on paper as well as
1
electronic forms. Assistive technology (AT) also provides a means for producing written information. Each person’s unique visual and learning needs must be considered when selecting the appropriate technology. The purpose of the Assistive Technology Assessment is to determine which AT tools are appropriate for meeting the person’s current and future needs. Conducting a comprehensive AT assessment should consider the range of tasks the person will be expected to complete in order to be successful in all areas of the educational program and prepare them for independent living and job/career success.
1.3
Proposed Work The proposed work wishes to bridge the gap between the inability of
the blind to decipher what is going on in the surroundings and the technology. The use of technology by the blind and visually impaired has been a mess of clunky software and chunky hardware. Screen readers, which read everything on a webpage including URLs and date stamps. But, finally, assistance technology is catching up. Artificial intelligence is powering rapid developments in computer vision and voice recognition, taking much-needed accessibility features mainstream. In doing so, people with disabilities are able to achieve more. Therefore, the proposed work wishes to see the world around you at the push of a button, seize the freedom to be active and do more of the things you love and move freely with seamless access to peripheral vision and virtually no lag.
2017-18
Dept. Of Electronics and Communication Engineering
Page 2
Chapter 2
Literature Survey Some of the ground breaking innovations to help the blind are existing in the market and are popular to a great extent.
1.JAWS, Job Access With Speech, is the world’s most popular screen reader, developed for computer users whose vision loss prevents them from seeing screen content or navigating with a mouse. JAWS provides speech and Braille output for the most popular computer applications on your PC. It has two multi-lingual synthesizers: Eloquence and Vocalizer Expressive, talking installation, convenient OCR feature for image files or inaccessible PDF documents. It supports PEARL Camera for direct access to Print documents or books and built-in free DAISY Player and full set of DAISY-formatted basic training books. It works with Microsoft Office, Google Docs, Chrome, Internet Explorer, Firefox, Edge, and much more. It also supports Windows 10, Windows 8.1, and Windows 7, including touch screens and gestures. Saves time with Skim Reading and Text Analyzer and fast information look-up at your fingertips with Research. It is fully compatible with ZoomText, Fusion, MAGic, and the OpenBook Scanning and Reading Software.
3
2.Magnification software works similarly to a high-powered magnifying glass moving over a page. They can magnify all screen items by following the mouse cursor or keyboard. Compatible with most Windows operating systems. Mac computers have a built-in magnification function. It is possible to use screen magnification software in conjunction with a screen reader for individuals who need both types of technology. Examples of screen-magnification programs include ZoomText and Magic. Some low-vision individuals might be able to benefit from larger monitors and internal features that enlarge font, increase contrast or otherwise modify computer functions. However, those features are not adequate for many low-vision Individuals and additional magnification software is necessary.
3.Refreshable Braille displays provide access to information on a computer screen by electronically raising and lowering different combinations of pins in braille cells. A braille display can show up to 80 characters from the screen and is refreshablethat is, it changes continuously as the user moves the cursor around on the screen, using either the command keys, cursor routing keys, or Windows and screen reader commands. The braille display sits on the users desk, often underneath the computer keyboard. The advantages of braille displays over synthetic speech are that it provides direct access to information; allows the user to check format, spacing, and spelling; and is quiet. Later models of braille displays offer note-taking and file storage capabilities. Some personal digital assistants (PDAs) may be connected to a computer and used as a braille display or speech synthesizer. Typically, braille displays are available with 40, 70 or 80 characters.
2017-18
Dept. Of Electronics and Communication Engineering
Page 4
Figure 2.1: Chart
2017-18
Dept. Of Electronics and Communication Engineering
Page 5
Figure 2.2: Refreshable Braille Display
2017-18
Dept. Of Electronics and Communication Engineering
Page 6
Chapter 3
Conceptual Design 3.1 3.1.1
Theoretical background SIGHT-Document Reader for Impaired Vision
This portable device can be used in many applications in robotics, automation, hobby projects and more. For example, you can focus your webcam to a text, such as English alphabets, on a signboard, followed by pressing a pushbutton switch connected to Raspi. It will capture the text and convert it to speech and read it out aloud to you. When you get bored of reading books, just click a picture of the textbook page and make it read the same aloud to you.The system uses a webcam, Raspberry Pi and a pushbutton switch.
3.2 3.2.1
Tools Used RASPBERRY PI 3
1.RASPBERRY PI 3 is the third generation pi.the RASPBERRY PI is a low cost single board packing considerable computer power in a size of a credit card. the RASPBERRY PI board contains many features like camera connector,ethernet port GPIO pins for interfacing sensors and switches, USB ports to connect to
7
external devices like keyboard mouse,Wi-Fi adapter etc.HDMI port to interface to monitors like LCD screens projectors. All models of RASPBERRY PI includes an ARM compatible CPU and an on-chip GPU.
Figure 3.1: RASPBERRY PI 3
3.2.2
Raspberry Pi Camera
2.The Raspberry Pi Camera Board plugs directly into the CSI connector on the Raspberry Pi. It’s able to deliver a crystal clear 5MP resolution image, or 1080p HD video recording.The Raspberry Pi Camera Module is a 5MP CMOS camera with a fixed focus lens that is capable of capturing still images as well as high definition video. Stills are captured at a resolution of 2592 x 1944, while video is supported at 1080p at 30 FPS, 720p at 60 FPS and 640x480 at 60 or 90 FPS.
Figure 3.2: Raspberry Pi Camera
2017-18
Dept. Of Electronics and Communication Engineering
Page 8
3.2.3
TESSERACT OCR
3.Tesseract is an optical character recognition engine for various operating systems.it is an OCR engine with support for unicode and the ability to recognize more than 100 languages out of the box. It can be trained to recognize other languages.Tesseract is used for text detection on mobile devices, in video, and in Gmail image spam detection.
3.2.4
FLITE TEXT TO SPEECH
4. Flite (festival-lite) is a small, fast open source text to speech synthesis engine developed at CMU and primarily designed for small embedded machines and/or large servers. Flite is designed as an alternative text to speech synthesis engine to Festival for voices built using the FestVox suite of voice building tools.
2017-18
Dept. Of Electronics and Communication Engineering
Page 9
Chapter 4
Methodology Planned 4.1 4.1.1
Steps Followed Operating System Setup and Configuration
Format an 8GB or larger microSD card with Raspbian Jessie Lite . You will need to access the Raspberry remotely via SSH. On Windows, you can use PUTTY SSH terminal program.
Insert the microSD card into the Pi and plug in the power now. To remotely log in to your Raspberry Pi, you will need to find its IP address.
Once logged in as pi user, update and upgrade all packages on your Raspberry Pi.Configure the Raspberry: In interfacing options, enable camera and force the audio through headphone jack only.
10
4.1.2
Application Software Installation
Install initial required software:
sudo apt install git -y
Create a folder with a test program as well as the final program. Run the test.sh script to ensure that the camera as well as the speaker is working well and is ready to use. Run the test program which sets the volume, plays some text-to-speech audio and takes a picture. If audio cannot be heard, the necessary troubleshooting can be carried out.
The test program saves a photo to test.jpg. You will need to copy this image over to a PC so that you can see the focus and the field of view. A quick and dirty way to do this is to start a tiny web server on your Pi and use a browser
python -m SimpleHTTPServer 8080
Then browse to http://IPaddress:8080/
4.1.3
Hardware Construction and Raspberry Pi Connections
A considerably large wooden box can be used for this purpose since the camera stays in place and hence the focus of the image stays intact. Drill a hole on the front right corner for the momentary button. Drill holes for p ower source, speakers and a slit for the Raspberry Pi camera cable.
Mount a flat wood strip on the back of the box lid vertically. Mount another flat wood strip on top of the vertical. The height of the camera determines 2017-18
Dept. Of Electronics and Communication Engineering
Page 11
the size of the document and the focus needed. The strip should be higher for larger area documents. Mount the camera facing downward from the end of the horizontal wood strip.
Install the momentary button and connect wires between it and Pi GPIO pins 24 and GND. Also, connect the buttons LED through a 220 ohm resistor to Pi GPIO pins 18 and GND.
4.1.4
Testing and Operation
Run the final code PiTextReader.py on the Raspberry Pi.
When you boot the Pi, you will hear a OK Ready as well as see the button LED light. Anytime the LED is lit, the unit is ready to go.
Put some printed text under the camera, preferably just a few lines of black text on white paper. Press the button. The LED should light and a camera click sound as well as speech OK working should sound. After a few seconds, the text should be read. If the text is distorted, too dark or too light, sideways or upside down, then the result will be an unintelligible or unclear speech.
It can take between 5-20 seconds to convert and start reading. If you need to stop reading, you can press the button while the audio is still playing (the LED is off.) Once the speech is completed, after a couple seconds, the LED comes back on and you will hear OK Ready again. Its ready to take another scan.
2017-18
Dept. Of Electronics and Communication Engineering
Page 12
Chapter 5
Detailed Design 5.1
Block diagram of SIGHT:Document Reader for Impaired Vision
The below figure shows the block diagram of the proposed work.
13
Figure 5.1: Block diagram of IOT based surveillance camera
5.2
Circuit of the SIGHT-Document Reader for Impaired Vision
The below is the circuit diagram for Raspberry pi connections to the servo driver as well as to the servo motors.It describes the flow from the Internet to drive the motors
2017-18
Dept. Of Electronics and Communication Engineering
Page 14
Figure 5.2: Circuit diagram
2017-18
Dept. Of Electronics and Communication Engineering
Page 15
5.3
Working of the circuit The proposed system includes hardware components such as Raspberry
Pi, a Pi camera and a push button switch only. The python program written is configured to run each time the Raspberry Pi boots. When the Pi is powered on, you hear a ”OK Ready” indicating that it is ready to capture an image so as to convert it. Once, the image is captured, it is stored as ”image.jpg”. Tesseract OCR converts this image into text. This might take a span of around 5 to 20 seconds depending on the length of the text. If at any time , you wish to stop reading the text, the button is to be pressed. The program is written so that on the push of the button ,GPIO 24 pin goes LOW, that is, it is connected to the ground. Hence, the text to speech conversion is stopped via the code. After the push of the button, there is a delay of a few seconds and then you hear a ”OK Ready” once again indicating the system’s readiness to capture the next image.
2017-18
Dept. Of Electronics and Communication Engineering
Page 16
Chapter 6
Results The above circuit after implementation converted text into speach output through the speaker. Using tesseract OCR and Flite text to speach. When the Pi reboots image captured is stored as image.jpg. Tesseract OCR converts this image into a text file image.txt and Flite speaks the text in the file via speaker.
6.1
Expected results
Output Analysis: 1.When you boot the Pi, you will hear a OK Ready as well as see the button LED light. Anytime the LED is lit, the unit is ready to go. 2.Press the button. The LED should light and a camera click sound as well as speech OK working should sound. After a few seconds, the text should be read. If the text is distorted, too dark or too light, sideways or upside down, then the result will be unclear speech. 3.If you need to stop reading, you can press the button while the audio is still playing.
17
6.2
SIGHT-Document Reader for Impaired Vision
Figure 6.1: camera
2017-18
Dept. Of Electronics and Communication Engineering
Page 18
6.3
Budget Analysis
The overall budject of the design is Rs3800.
Figure 6.2: Budget for components
2017-18
Dept. Of Electronics and Communication Engineering
Page 19
Chapter 7
Future scope and conclusion 7.1
Future Scope
The project can be extended by adding suitable features like, a) The speakers can be replaced with bone conductive earphones which are more portable and easily operated. b) The project can be modified to read specific text within the image ignoring unimportant text or any other disturbances in the image. c) We can integrate the system along with glasses and make it suitable to recognize everyday objects. d) It can be modified to recognize various fonts and also be suitable for languages other than english.
7.2
Conclusion In this project we have designed a prototype system to read printed text
for assisting the blind. Off the shelf OCR is used to perform word recognition and transform into audio output for the users. As the raspberry Pi board is powered, the camera captures an image. Using tesseract library the image will
20
be converted into data and the data detected from the image will be pronunced through the speakers using Flite library.
The implementation of SIGHT approach has provided a real time monitoring of the target area.
2017-18
Dept. Of Electronics and Communication Engineering
Page 21
Chapter 8
References 1 https://www.hackster.io/rgrokett/pitextreader-for-impaired-vision-695331/ 2 https://electronicsforu.com/electronics-projects/hardware-diy/see-speak-usingraspberry-pi/ 3 https://www.instructables.com/id/HOW-TO-INSTALL-RASPBIAN-OSIN-YOUR-RASPBERRY-PI/
22
