DECLARATION I hereby hereby declare declare that that the result resultss embodie embodied d in this this disser dissertati tation on entitle entitled d “CAR NUMBER PLATE RECOGNITION” is carried out by me during the year 2008-09 in
partial partial fulfillment fulfillment of the requirements requirements of the award of degree, degree, Bachelor Bachelor of Engineering in Electronics and Communication Engineering from M.V.S.R Engineering College, affiliated to Osmania University, Hyderabad. I have not been submitted the same to any other university or organization for award of any other degree or diploma.
D. Keerthi
K. Aruna Kumari
B. Spandana
ACKNOWLEDGEMENTS The satisfaction and euphoria that accompany the successful completion of any task would be, but incomplete without the mention of the people who made it possible, whose constant guidance and encouragement have crowned our efforts with success. We expr expres esss our our sinc sincer eree than thanks ks and and grat gratit itud udee to our our exte extern rnal al guid guidee Mr. K. Rambabu Director of Laser systems (RESEARCH CENTRE IMARAT).
We are extremely extremely indebted indebted to our internal internal guide, guide, Mr. Sudhir Dakey for his valuable help, and would like to express our profound gratitude towards his for providing all facilities to implement this project in the college and for his sagacious guidance through out the project in the college and also through out our college period. We would like express our deep felt gratitude to Dr. P.A. Sastry, Principal of M.V.S.R Engineering College for his kind co-operation and valuable suggestions. We would like to take this opportunity to express our heart-felt gratitude to our H.O.D. Dr. B. Hari Kumar for providing the necessary infrastructure to complete this project. project. We also thank the staff of the ECE department who helped us morally for the successful completion of this project.
RCI PROFILE The Research Centre Imarat (RCI) was established by A.P.J. Abdul Kalam in 1988 on a campus campus 8 km from Defence Defence Research Research and Developme Development nt Laborat Laboratory ory [DRDL] [DRDL] at Kanchanbagh, Hyderabad. The center's state-of-the-art facilities are dedicated to work in advanced missile technologies. The Research Center Imarat complex is primarily associated with research and devel developm opmen entt of techn technol olog ogie iess for for miss missile iles, s, aircr aircraft aft and other other advanc advanced ed weap weapons ons,, Development activities at RCI include the Light Combat Aircraft as well as the missile programs. programs. The research research conducted conducted at RCI in areas such as materials, materials, electronics electronics and software provided the base on which an ambitious missile program could be successfully constructed. Work conducted at RCI also includes specialties such as automated test systems, data acquisition, analysis, control electronics, digital signal processing, quality control of high reliability electronic packages, gyro systems, environmental testing, and integration of multistage multistage aerospace aerospace vehicles. vehicles. Activities Activities range from software software operations, operations, navigational navigational checks, checks, sensor sensor interfac interfacee and calibra calibration tion,, to environm environmenta entall and integr integrati ation on check-ou check-outs. ts. Typical projects include development of a data acquisition system for surface to air missile ground segment. The RCI is one of the nodal laboratories that are involved in developing lightweight weight compos composite ite materi material al for the missil missiles es develop developed ed by the Defence Defence Research Research and Development Organization. RCI's Composite Product and Development Centre (CPDC) is also working on uses of high-tech material for social applications, such as light-weight artificial limbs.
ABSTRACT Car Car Numb Number er Plate Plate Reco Recogn gnit itio ion n System System means means to extra extract ct the the number number plate plate characters in ASCII format from the image. The aim of the project is to provide traffic management, control and other related applications This system takes a vehicle image of any size breaks it into smaller image image pieces. These pieces are then analyzed to locate the exact location of number plate in the image. Once the area of the number plate (its x and y coordinates) is found the plate is parsed to extract the character from it. These characters are then given to the Optical Character Recognition Recognition (OCR) module. module. OCR program recognizes recognizes those those characters and converts them to ASCII format. The technique used in finding the exact location of number plate in the vehicle image is signature signature technique. technique. The plate plate thus obtained obtained is is given to to the OCR OCR module. module.
The
goal of Optical Character Recognition Recognition is to classify classify optical patterns (often contained in a digital image) corresponding to alphanumeric or other characters. The process of OCR involves involves several several steps steps includi including ng segmen segmentati tation, on, feature feature extract extraction ion,, and classif classificat ication ion.. Templates are 2D correlated to extract the characters.
CAR NUMBER PLATE RECOGNITION
CONTENTS LIST OF FIGURES
iv
LIST OF TABLES
vi
I. INTRODUCTION
1
1.1 Introduction
1
1.2 Aim Of The Project
2
1.3 History
3
1.4 Working
3
1.5 Algorithm
4
1.6 Conclusion
4
II. IMPLEMENTATION TOOL – MATLAB
5
2.1 Introduction
5
2.2 MATLAB
5
2.3 MATLAB Systems
6
2.4 Graphics and Visualization
9
2.5 Features
9
2.6 M files
11
2.7 Images in MATLAB
11
2.8 Working with Images in MATLAB
12
2.9 Graphical User Interface
12
2.10 GUI Working
13 i
2.11 Defining User Interface Controls
14
2.12 Guide: An Overview
23
2.13 Guide Tools Summary
24
2.14 Conclusion
24
III. PROJECT DESCRIPTION
25
3.1 Introduction
25
3.2 Digital Image
25
3.3 Digital Image Processing
25
3.4 Key Steps Involved in the Project
26
3.5 Techniques Used In the Project
27
3.6 Signature Technique
27
3.7 Optical Character Recognition (OCR)
28
3.8 Template Matching
29
3.9 Conclusion
31
IV. WORKING DESCRIPTION
32
4.1 Introduction
32
4.2 Algorithm
32
4.3 Flowchart Depicting the Operations of the System
33
4.4 Flow of Function Calls in the program
34
4.5 MATLAB Code
35
4.6 GUI Code
46 ii
4.7 Conclusion
51
V. ADVANTAGES, DISADVATAGES AND APPLICATIONS
52
5.1 Advantages
52
5.2 Disadvantages
52
5.3 Applications
53
VI. RESULTS & CONCLUSION
57
6.1 Results
57
6.2 Conclusion
63
6.3 Future Advancements
63
APPENDIX A
64
BIBLIOGRAPHY
76
LIST OF FIGURES 2.1 MATLAB Development Environment
7
2.2 Simple GUI
13
2.3 Property Inspector
14
2.4 Push Button
15
iii
2.3 Push Button with Image
16
2.6 Slider
16
2.7 Radio Button
17
2.8 Check Box
18
2.9 Edit Text Box
19
2.10 Static Text
20
2.11 Pop-up Menu
20
2.12 Toggle Button
21
2.13 Axes
22
2.14 Layout Editor
24
3.1 Segmented Character Of The Number Plate
30
3.2 Template Image
30
3.3 Result Of Correlation
31
4.1 Flowchart Depicting The Operations Of The System
33
4.2 Flow Of Function calls In The Program
34
6.1 Input Image To Tmain
57
6.2 Binarised Image Of Input Image
58
6.3 Row wise Segmentation Of The Binarised Image
59
6.4 High Frequency Areas Of The Image
60
6.5 Number Plate From The Input Image
61
6.6 Segmented Characters Of The Number Plate
62
A.1 Pixel Coordinate System
66 iv
A.2 Spatial Coordinate System
67
A.3 Binary Image
69
A.4 Intensity Image
70
A.5 Indexed Image
71
A.6 RGB Image
72
LIST OF TABLES
8.1 Data Classes Supported By IPT Functions
68
8.2 Image Types And Data classes
73
8.3 Image Type conversion Functions
74
v
CHAPTER I INTRODUCTION 1.1 INTRODUCTION
“Car Number Plate Recognition” system uses Optical Character Recognition
(OCR) software to convert images of vehicle registration numbers into information for real time or retrospective matching with law enforcement and other databases. This system is an integrated hardware-software device that reads the vehicles number plate and outputs the number plate number in ASCII - to some data processing system. This system is also known as License Plate Recognition (LPR) and Automatic License Plate Recognition (ALPR) systems. It is believed that there are currently more than half a billion cars on the roads worldwide. All those vehicles have their Vehicle Identification Number as their primary identifier. The vehicle identification number is actually a license number which states a legal license to participate in the public traffic. All vehicle world-wide should have its license number - written on a license plate - mounted onto its body (at least at the back side) and no vehicle without properly mounted, well visible and well readable license plate should run on the roads. The license number is the most important identification data a computer system should treat when dealing with vehicles. If the data is already in the computer most of these tasks are rather easy to be carried out. Suppose a company's security manager would require a system that precisely tells at every moment where the cars of the company are: in the garage or out on roads. By registering every single drive-out from and drive-in to the garage, the system could always tell which car is out and which is in. The key issue of this task is that the registration of the movement of the vehicles should be done automatically by the system, otherwise it would require manpower. Concisely, Car Number Plate Recognition is automation of data input, where data equals the registration number of the vehicle. This system replaces, redeems the task of manually typing the plate number of the bypassing vehicle into the computer system. 1
1.2 AIM OF THE PROJECT The aim of the project is to develop an efficient Car Number Plate Recognition System for necessary applications. The scope of the project is broadly categorized into modules as follows: Number Plate Recognition Templates Templates Management
1.3 HISTORY The Car Number Plate Recognition System was invented in 1976 at the Police Scientific Development Branch in the UK. Prototype systems were working by 1979 and contracts were let to produce industrial systems, first at EMI Electronics then at Computer Recognition Systems (CRS) in Wokingham, UK. Early trial systems were first used in Britain in 1979 with trial units placed on the A1 road and the Dartford tunnel. To minimize errors, license plate's character font, positioning and sizes were optimized for the character recognition systems used. The first arrest due to a detected stolen car was made in 1981. Early CNPR systems were unable to read white or silver lettering on black background, as permitted on UK vehicles built prior to 1973. There are a number of other possible difficulties that the software must be able to cope with. These include: •
Poor image resolution, usually because the plate is too far away but sometimes resulting from the use of a low-quality camera.
•
Blurry images, particularly motion blur
•
Poor lighting and low contrast due to overexposure, reflection or shadows
•
An object obscuring (part of) the plate, quite often a tow bar, or dirt on the plate
•
A different font, popular for vanity plates (some countries do not allow such plates, eliminating the problem)
1.4 WORKING 2
As the vehicle approaches the secured area, and starts the cycle by stepping over a magnetic loop detector (which is the most popular vehicle sensor). The loop detector senses the presence of the car.
FIGURE 1.1
Now the illumination is activated and the camera (shown in the left side of the gate) takes pictures of the front or rear plates. The images of the vehicle include the plate.
FIGURE 1.2
Our project (CNPR System) reads the pixel information of this vehicle image and breaks it into smaller image pieces. These pieces are then analyzed to locate the exact location of number plate in the image. Once the area of the number plate (its x and y coordinates) is found the plate is parsed to extract the character from it. These characters are then given to the OCR module. OCR program recognizes those characters and coverts them to text format.
3
FIGURE 1.3
Once the car number was obtained it can be used according to the application.
1.5 ALGORITHM
This system takes a vehicle image of any size.
Breaks it into smaller image pieces.
These pieces are then analyzed to locate the exact location of number plate in the image.
Once the area of the number plate (its x and y coordinates) is found the plate is parsed to extract the character from it.
These characters are then recognized and converted to ASCII text format.
1.6 CONCLUSION There are frequent situations in which a system able to recognize registration numbers. Our project deals with the identification of the characters from the image of the car (containing number plate). The basic ideas of the project are discussed in brief.
CHAPTER II IMPLEMENTATION TOOL - MATLAB 2.1 INTRODUCTION This chapter discusses the implementation software, MATLAB used in the project. This project is also implemented in GUI (Graphical User Interface). This chapter also discusses main features of GUI.. 4
2.2 MATLAB MATLAB is a high-performance language for technical computing. It integrates computation, visualization, and programming in an easy-to-use environment where problems and solutions are expressed in familiar mathematical notation. Typical uses include: •Math and computation •Algorithm development •Modeling, simulation, and prototyping •Data analysis, exploration, and visualization •Scientific and engineering graphics •Application development, including graphical user interface building
MATLAB is an interactive system whose basic data element is an array that does not require dimensioning. This allows you to solve many technical computing problems, especially those with matrix and vector formulations, in a fraction of the time it would take to write a program in a scalar noninteractive language such as C or FORTRAN. The name MATLAB stands for Matrix Laboratory. MATLAB was originally written to provide easy access to matrix software developed by the LINPACK and EISPACK projects. Today, MATLAB uses software developed by the LAPACK and ARPACK projects, which together represent the state-of-the-art in software for matrix computation. MATLAB has evolved over a period of years with input from many users. In university environments, it is the standard instructional tool for introductory and advanced courses in mathematics, engineering, and science. In industry, MATLAB is the tool of choice for high-productivity research, development, and analysis. MATLAB features a family of add-on application-specific solutions called toolboxes. Very important to most users of MATLAB, toolboxes allow you to learn and apply specialized technology. Toolboxes are comprehensive collections of MATLAB functions (M-files) that extend the MATLAB environment to solve particular classes of
5
problems. Areas in which toolboxes are available include image processing, signal processing, control systems, neural networks, fuzzy logic, wavelets, simulation, and many others.
2.3 MATLAB SYSTEM The MATLAB system consists of five main parts:
Development Environment
MATLAB Mathematical Function Library
MATLAB Language
Handle Graphics
MATLAB Application Program Interface (API).
Development Environment
This is the set of tools and facilities that help you use MATLAB functions and files. Many of these tools are graphical user interfaces. It includes the MATLAB desktop and Command Window, a command history, and browsers for viewing help, the workspace, files, and the search path.
6
FIGURE 2.1 MATLAB Development Environment
MATLAB Mathematical Function Library
This is a vast collection of computational algorithms ranging from elementary functions like sum, sine, cosine and complex arithmetic, to more sophisticated functions like matrix inverse, matrix eigen values, Bessel functions and Fourier transforms.
7
MATLAB language
This is a high-level matrix/array language with control flow statements, functions, data structures, input/output, and object-oriented programming features. It allows both “programming in the small” to rapidly create quick and dirty throw-away programs, and “programming in the large” to create complete large and complex application programs. Handle Graphics
This is the MATLAB graphics system. It includes high-level commands for twodimensional and three-dimensional data visualization, image processing, animation, and presentation graphics. It also includes low-level commands that allow you to fully customize the appearance of graphics as well as to build complete graphical user interfaces on your MATLAB applications. MATLAB Application Program Interface (API)
This is a library that allows you to write C and FORTRAN programs that interact with MATLAB. It include facilities for calling routines from MATLAB (dynamic linking), calling MATLAB as a computational engine, and for reading and writing MAT-files. Our Image processing laboratory based on MATLAB has the capability to perform the following tasks: 1. Perform algebraic operations such as addition, subtraction, multiplication, and division. These operations can be used to perform operations on images such as noise reduction by using averages, movement detection, and algebraic masking. 2. Geometric transformations such as translation, rotation, and scaling 3. Space domain operations such as histogram modification (scaling, offset, amplitude change) non linear point operations (absolute value, squaring, square root, log scale compression, edge detection) 4. Binary image processing (thresholding, logic operations). 5. Non linear image processing such as morphologic operators (opening, closing). Structuring element choice. Dilation, erosion. 8
6. Frequency domain processing. Fourier transform, log compression. 7. Filtering by linear convolution. Filter design (low pass, high pass, band pass, band reject.). Gaussian Filters. Linear restoration. White noise non linear filtering. 8. Digital Image coding and compression. Compression measures, losses compression, entropy, optimal coding.
2.4 GRAPHICS & VISUALIZATION MATLAB has extensive facilities for displaying vectors and matrices as graphs, as well as annotating and printing these graphs. It includes high-level functions for two dimensional and three-dimensional data visualization, image processing, animation, and presentation graphics. It also includes low-level functions that allow you to fully customize the appearance of graphics as well as to build complete graphical user interfaces on your MATLAB applications.
2.5 FEATURES Variables
A MATLAB variable is essentially a tag that you assign to a value while that value remains in memory. Variables are defined with the assignment operator, '='. For instance, x=17 defines a variable named x and assigns it a value of 17. Values can come from literal values such as string constants or numeric immediates, or from the values of other variables, or from the output of a function. Matrices
Essentially, the only data objects in MATLAB are rectangular numerical matrices. There are no restrictions placed on the dimensions of a matrix (except by system resources), but special meaning is sometimes attached to 1*1 matrices (scalars) and matrices with only one row or column (vectors). The memory required for the storage of each matrix is automatically allocated by MATLAB. 9
The easiest way to enter a matrix into MATLAB is to provide an explicit list of elements enclosed in square brackets [ ]. MATLAB uses the following conventions: -
A matrix element can be any valid MATLAB expression.
-
Matrix elements are separated by blanks or commas.
-
A semicolon or carriage return is used to indicate the end of a row.
Operators
Relational Operators Logical Operators <
Less than | And
<= Less than or equal to & or >
Greater ~ Not
>= Greater or equal to == Equal to ~= Not equal to Strings
String is an array of characters. Each character is represented internally by its ASCII value. This is an example of a string str = 'I am learning MATLAB this semester.'
2.6 M-FILES We can create our own matrices using M-files, which are text files containing MATLAB code. Use the MATLAB Editor or another text editor to create a file containing
10
the same statements you would type at the MATLAB command line. Save the file under a name that ends in .m. For example, create a file containing these five lines. A = [... 16.0 3.0 2.0 13.0 5.0
10.0 11.0 8.0
9.0
6.0 7.0
4.0
15.0 14.0 1.0];
12.0
Store the file under the name magik.m. Then the statement magik reads the file and creates a variable A, containing our example matrix.
2.7 IMAGES IN MATLAB The basic data structure in MATLAB is the array, an ordered set of real or complex elements. This object is naturally suited to the representation of images, real-valued ordered sets of color or intensity data. MATLAB stores most images as two-dimensional arrays (i.e., matrices), in which each element of the matrix corresponds to a single pixel in the displayed image. (Pixel is derived from picture element and usually denotes a single dot on a computer display.) For example, an image composed of 200 rows and 300 columns of different colored dots would be stored in MATLAB as a 200-by-300 matrix. Some images, such as RGB, require a three-dimensional array, where the first plane in the third dimension represents the red pixel intensities, the second plane represents the green pixel intensities, and the third plane represents the blue pixel intensities. This convention makes working with images in MATLAB similar to working with any other type of matrix data, and makes the full power of MATLAB available for image processing applications. For example, you can select a single pixel from an image matrix using normal matrix subscripting. I (2, 15) This command returns the value of the pixel at row 2, column 15 of the image I.
2.8 WORKING WITH IMAGES IN MATLAB
11
Images are most commonly stored in MTALAB using the logical, unit 8, unit 16 and double data types (Detailed information about data types is given in Appendix). We can perform many standard MATLAB array manipulations on uint8 and uint16 image data, including
Basic arithmetic operations, such as addition, subtraction, and multiplication.
Indexing, including logical indexing.
Reshaping, reordering, and concatenating.
Reshaping, reordering, and concatenating.
Using relational operators. Certain MATLAB functions, including the find, all, any, conv2, convn, fft2, fftn,
and sum functions, accept uint8 or uint16 data but return data in double-precision format.
2.9 GRAPHICAL USER INTERFACE (GUI) A graphical user interface (GUI) is a graphical display that contains devices, or components, that enable a user to perform interactive tasks. To perform these tasks, the user of the GUI does not have to create a script or type commands at the command line. Often, the user does not have to know the details of the task at hand. The GUI components can be menus, toolbars, push buttons, radio buttons, list boxes, and sliders— just to name a few. In MATLAB, a GUI can also display data in tabular form or as plots, and can group related components. The following fig 2-1 illustrates a simple GUI. The GUI contains
-
An axes component -
A pop-up menu listing three data sets that correspond to MATLAB functions:
peaks, membrane, and sinc -
A static text component to label the pop-up menu
-
Three buttons that provide different kinds of plots: surface, mesh, and contour 12
FIGURE 2.2 Simple GUI
2.10 WORKING OF GUI Each component, and the GUI itself, are associated with one or more user-written routines known as callbacks. The execution of each callback is triggered by a particular user action such as a button push, mouse click, selection of a menu item, or the cursor passing over a component. We, as the creator of the GUI, provide these callbacks. In the GUI described in “What Is a GUI?” the user selects a data set from the pop-up menu, then clicks one of the plot type buttons. Clicking the button triggers the execution of a callback that plots the selected data in the axes. This kind of programming is often referred to as event-driven programming. The event in the example is a button click. In event-driven programming, callback execution is asynchronous, controlled by events external to the software. In the case of MATLAB GUIs, these events usually take the form of user interactions with the GUI. The writer of a callback has no control over the sequence of events that leads to its execution or, when the callback does execute, what other callbacks might be running simultaneously.
2.11 DEFINING USER INTERFACE CONTROLS User interface controls include push buttons, toggle buttons, sliders, radio buttons,
13
edit text controls, static text controls, pop-up menus, check boxes, and list boxes. To define user interface controls, you must set certain properties. To do this: Use the Property Inspector to modify the appropriate properties. Open the Property Inspector by selecting Property Inspector from the View menu. In the layout area, select the component you are defining.
FIGURE 2.3 Property Inspector
Subsequent topics describe commonly used properties of user interface controls and offer a simple example for each kind of control: Push Button
To create a push button with label Button 1, as shown in this figure
14
FIGURE 2.4 Push Button
Specify the push button label by setting the String property to the desired label, in this case, Button 1. To display the character in a label, use two characters in the string. The words remove, default, and factory (case sensitive) are reserved. To use one of these as a label, prepend a backslash (\) to the string. For example, \remove yields remove. The push button accommodates only a single line of text. If we specify more than one line, only the first line is shown. If we create a push button that is too narrow to accommodate the specified String, MATLAB truncates the string with an ellipsis. If we want to set the position or size of the component to an exact value, then modify its Position property. To add an image to a push button, assign the button’s CData property an m-by-n by-3 array of RGB values that defines a true color image. You must do this programmatically in the opening function of the GUI M-file. For example, the array img defines a 16-by-64-by-3 true color image using random values between 0 and 1 (generated by rand). img = rand (16, 64, 3); set (handles.pushbutton1,'CData', img); where pushbutton1 is the push button’s Tag property.
FIGURE 2.5 Push Button with Image
Slider
To create a slider as shown in this figure 15
FIGURE 2.6 Slider
Specify the range of the slider by setting its Min property to the minimum value of the slider and its Max property to the maximum value. The Min property must be less than Max. Specify the value indicated by the slider when it is created by setting the Value property to the appropriate number. This number must be less than or equal to Max and greater than or equal to Min. If we specify Value outside the specified range, the slider is not displayed. Control the amount the slider Value changes when a user clicks the arrow button to produce a minimum step or the slider trough to produce a maximum step by setting the Slider Step property. Specify Slider Step as a two-element vector, [min_step, max_step], where each value is in the range [0, 1] to indicate a percentage of the range.
Radio Button
To create a radio button with label Indent nested functions , as shown in this figure
FIGURE 2.7 Radio Button
Specify the radio button label by setting the String property to the desired label, in this case, Indent nested functions. 16
To display the & character in a label, use two & characters in the string. The words remove, default, and factory (case sensitive) are reserved. To use one of these as a label, prepend a backslash (\) to the string. For example, \remove yields remove. The radio button accommodates only a single line of text. If you specify more than one line, only the first line is shown. If we create a radio button that is too narrow to accommodate the specified String, MATLAB truncates the string with an ellipsis. Create the radio button with the button selected by setting its Value property to the value of its Max property (default is 1). Set Value to Min (default is 0) to leave the radio button unselected. Correspondingly, when the user selects the radio button, MATLAB sets Value to Max. MATLAB sets Value to Min when the user deselects it.
Check Box
To create a check box with label Display file extension that is initially checked, as shown in this figure
FIGURE 2.8 Check Box
Specify the check box label by setting the String property to the desired label, in this case, Display file extension.
17
To display the & character in a label, use two & characters in the string. The words remove, default, and factory (case sensitive) are reserved. To use one of these as a label, prepend a backslash (\) to the string. For example, \remove yields remove. The check box accommodates only a single line of text. If we specify a component width that is too small to accommodate the specified String, MATLAB truncates the string with an ellipsis. Create the check box with the box checked by setting the Value property to the value of the Max property (default is 1). Set Value to Min (default is 0) to leave the box unchecked. Correspondingly, when the user clicks the check box, MATLAB sets Value to Max when the user checks the box and to Min when the user unchecks it.
Edit Text
To create an edit text component that displays the initial text Enter your name here, as shown in this figure
FIGURE 2.9 Edit Text Box
Specify the text to be displayed when the edit text component is created by setting the String property to the desired string, in this case, Enter your name here. 18
To display the & character in a label, use two & characters in the string. The words remove, default, and factory (case sensitive) are reserved. To use one of these as a label, prepend a backslash (\) to the string. For example, \remove yields remove. To enable multiple-line input, specify the Max and Min properties so that their difference is greater than 1. For example, Max = 2, Min = 0. Max default is 1, Min default is 0. MATLAB wraps the string and adds a scroll bar if necessary. If Max-Min is less than or equal to 1, the edit text component admits only a single line of input. If you specify a component width that is too small to accommodate the specified string, MATLAB displays only part of the string. The user can use the arrow keys to move the cursor over the entire string.
Static Text
To create a static text component with text Select a data set , as shown in this figure
FIGURE 2.10 Static Text
Specify the text that appears in the component by setting the component String property to the desired text, in this case Select a data set. To display the & character in a list item, use two & characters in the string. The words remove, default, and factory (case sensitive) are reserved. 19
To use one of these as a label, prepend a backslash (\) to the string. For example, \remove yields remove. If our component is not wide enough to accommodate the specified String, MATLAB wraps the string. Pop-Up Menu
To create a pop-up menu (also known as a drop-down menu or combo box) with items one, two, three, and four, as shown in this figure
FIGURE 2.11 Pop-up Menu
To display the & character in a menu item, use two & characters in the string. The words remove, default, and factory (case sensitive) are reserved. To use one of these as a label, prepend a backslash (\) to the string. For example, \remove yields remove. If the width of the component is too small to accommodate one or more of the specified strings, MATLAB truncates those strings with an ellipsis. To select an item when the component is created, set Value to a scalar that indicates the index of the selected list item, where 1 corresponds to the first item in the list. If you set Value to 2, the menu looks like this when it is created. Toggle Button
To create a toggle button with label Left/Right Tile, as shown in figure 20
FIGURE 2.12 Toggle Button
Specify the toggle button label by setting its String property to the desired label, in this case, Left/Right Tile. To display the & character in a label, use two & characters in the string. The words remove, default, and factory (case sensitive) are reserved. To use one of these as a label, prepend a backslash (\) to the string. For example, \remove yields remove. The toggle button accommodates only a single line of text. If you specify more than one line, only the first line is shown. If we create a toggle button that is too narrow to accommodate the specified String, MATLAB truncates the string with an ellipsis.
Axes
To create an axes as shown in the following figure
21
FIGURE 2.13 Axes
Allow for tick marks to be placed outside the box that appears in the Layout Editor. The axes above looks like this in the layout editor; placement allows space at the left and bottom of the axes for tick marks. Functions that draw in the axes update the tick marks appropriately. Use the title, xlabel, ylabel, zlabel, and text functions in the GUI M-file to label an axes component. For example, xlh = (axes_handle,'Years') labels the X-axis as Years. The handle of the X-axis label is xlh. The words remove, default, and factory (case sensitive) are reserved. To use one of these in component text, prepend a backslash (\) to the string. For example, \remove yields remove.
2.12 GUIDE: AN OVERVIEW GUI Layout
Guide, the MATLAB graphical user interface development environment, provides a set of tools for creating graphical user interfaces (GUIs). These tools simplify the process of laying out and programming GUIs. Using the GUIDE Layout Editor, you can populate a GUI by clicking and dragging GUI components—such as axes, panels, buttons, text fields, sliders, and so on—into the layout area. We can also create menus and context menus for the GUI. From the Layout Editor, you can size the GUI, modify component look and feel, align components, set tab order, view a hierarchical list of the component objects, and set GUI options. 22
GUI Programming
GUIDE automatically generates an M-file that controls how the GUI operates. This M-file provides code to initialize the GUI and contains a framework for the GUI callbacks —the routines that execute when a user interacts with a GUI component. Using the M-file editor, you can add code to the callbacks to perform the functions you want.
2.13 GUIDE TOOLS SUMMARY The GUIDE tools are available from the Layout Editor shown in the figure below. The tools are called out in the figure and described briefly below.
23
.
FIGURE 2.14 Layout Editor
2.14 CONCLUSION The project is implemented using the tool MATALAB. All the important features of the MATALAB are discussed. MATLAB provides toolboxes for various fields. We are implementing this project using Image Processing Toolbox. The detailed discussion about the toolbox is given in Appendix.
CHAPTER III PROJECT DESCRIPTION 3.1 INTRODUCTION 24
In the previous chapter, the tools used for implementing the project and their details are discussed. In this chapter we discuss the basics of digital image processing. It also discusses in detail the techniques used for implementing the project. Some important notes regarding number plate recognition system are also explained. The technique used is signature technique to find the location of number plate .Optical Character Recognition is used for recognizing the characters to get the characters in ASCII text format.
3.2 DIGITAL IMAGE Vision is the most advanced of our senses, so it is not surprising that images play the single most important role in human perception. However, unlike humans, who are limited to visual band of electromagnetic (EM) spectrum, imaging machines cover almost the entire EM spectrum, ranging from gamma to radio waves. They can operate also on images generated by sources that humans are not accustomed to associating with images. These include ultrasound, electron microscopy, and computer-generated images. 3.1.1 Definition
An image may be defined as a two dimensional function, f(x, y), where x and y are spatial coordinates and the amplitude of any pair of coordinates is called the intensity or gray level of the image at that point. When x, y and the amplitude of the image are all finite, discrete quantities, we call the image a Digital Image.
3.3 DIGITAL IMAGE PROCESSING (DIP) The field of digital image processing refers to processing the digital images by means of a digital computer. The digital image is composed of a finite number of elements, each of which has a particular location and value. These elements are referred to as picture elements, image elements, pels and pixels. Pixel is the term most widely used to denote the elements of a digital image. There is no general agreement regarding where image processing stops and other related areas, such as image analysis and computer vision start. Sometimes a distinction is made by defining image processing as a discipline in which both the input and output of a process are images. One useful paradigm is to consider three types of computerized processes in this continuum: low-, mid-, and high-level processes.
25
Low-level processes involve primitive operations such as preprocessing to reduce noise, contrast enhancement and image sharpening. A low-level process is characterized by the fact that both its inputs and outputs are images.
Mid-level processes on images involve tasks such as segmentation (partitioning an image into regions or objects), description of those objects to reduce them to a form suitable for computer processing, and classification (recognition) of individual objects. A mid-level process is characterized by the fact that its inputs generally are images, but its outputs are attributes extracted from those images (e.g., edges, contours, and the identity of individual objects).
Higher-level processing involves “making sense” of an ensemble of recognized objects, as in image analysis, and, at the far end of the continuum, performing the cognitive functions normally associated with human vision.
3.4 KEY STEPS INVOLVED IN THE PROJECT Car number plate recognition system has three main components in it. 1. Converting the Bitmap image to Binary image. 2. Breaking the image into smaller pieces of image which are high frequency areas of the original image 3. Choosing the number plate from the image pieces returned by the above module,
and parsing the plate to extract out the character part. 4. Recognizing the characters in the image pieces
3.5 TECHNIQUES USED IN THE PROJECT
Signature technique: To detect the number plate from the image and parsing it to
extract the characters.
26
OCR by 2D Correlation: recognize the extracted characters and covert them to
text format.
3.6 SIGNATURE TECHNIQUE Signature technique helps in locating high frequency areas. If the image is binarised then most of the detail is lost from the image, leaving our area of interest more prominent. Taking row wise or column wise signature of an image gives the information about the less detail and more detail areas of the image. So it becomes easy to find out the areas with high frequencies. 3.6.1 Finding Probable Number Plate in the Image
Once the image is binarised its row wise histogram (sum of white or black pixels in each row) or signature is taken to find out which number of rows is showing ridges. These ridges are basically high frequency areas and one of these ridges will definitely be a number plate. A threshold value is used to indicate the starting and ending point of the ridge. The best results were shown by taking the average of the minimum point and the median of the row signature as the threshold. Once the ridges in the row signature of the image are obtained column wise signature of those row ridges is calculated. This will further refine the candidate image by removing those columns from the row ridge which do not possess much detail. This is done by choosing the ridges our of the column histogram of the ridge area in the row histogram. 3.6.2 Recognizing Number Plate from the Candidate Images
The x and y coordinates of all the high frequency pieces which are the candidates of number plate are known. As we can see that on the number plate there would be four to ten characters. So each character will show a ridge in the row signature of the image piece, secondly most of the information is lost because of binarising the image so only number plate area will show maximum number of ridges. Now if we take the row wise histogram of those binarised pieces we can see that number plate image shows more number of ridges as compared to any other candidate image. So image with maximum number of ridges in its row signature is chosen as the number plate. 3.6.3 Extraction of characters
27
Then the same signature technique is applied to extract the numbers from the number plate image. The difference was in the threshold value. Because here we needed to pick each ridge in the histogram therefore the minimum value of the histogram was chosen as the thresholding value. And the reason is that all characters might not show ridges with equal peak (highest point in the ridge). Or a character like ‘X’ might be broken into two ridges. As it is obvious that the center of the character X will show very small peak. The extracted characters are then given to OCR module.
3.7 OPTICAL CHARACTER RECOGNITION (OCR) 3.7.1 What is OCR?
The goal of Optical Character Recognition (OCR) is to classify optical patterns (often contained in a digital image) corresponding to alphanumeric or other characters. The process of OCR involves several steps including segmentation, feature extraction, and classification. 3.7.2 Character Recognition: OCR by 2D Correlation
Description of method Given the chosen license plate and the coordinates that indicate where the characters are, we begin the OCR process by 2D correlation. We correlate each character with either the alphabet or the numeral templates then choose the value of each character based on the result of the correlation. The first three characters on the standard License Plates are alphabets; therefore, we correlate each one of them with the 26 alphabet templates. The latter four characters on the standard License Plates are numerals; therefore, we correlate each one of them with the 10 numeral templates. The result OCR is chosen based on the maximum values of the correlation for each character. However, we realized that certain characters are frequently confused. As a quick solution, we implanted a scheme to display possible alternatives. Those characters that are identified as easily misinterpreted are subjected to a correlation comparison with an array of other characters that is historically known to be easily confused with the originals. If there is a possibility that a letter or number can be confused with more than one character, the characters are listed in the output in the order of decreasing likelihood. This likelihood is based on the correlation 28
values of the character with the various templates, where high correlation denotes a good possibility. However, this scheme only occurs if the given letter does not have a very high correlation value (does not land above a nominal threshold). 3.7.3 Rationale
OCR by 2D correlation is the option that seems to strike the best balance between performance and difficulty in implementation. Details can be observed in the character isolation portion of the source code. 3.7.4. Possible problems/Weaknesses
OCR by 2D correlation is sensitive to the size of the license plate, which meant bigger or smaller alphabets and numbers in the picture. The 2-d correlation was very sensitive to this and frequently gave back wrong results due to different size license plates.
3.8 TEMPLATE MATCHING The steps for this process are: 1. Build a template for each of the letters to be recognized. A good first-approximation for a template is to the intersection of all instances of that letter in the number plate. However, more fine-tuning of this template must be done for good performance. 2. Erode the original image using this template as structuring element. All 1 pixels in the resulting image correspond to all matches found for the given template. 3. Find the objects in the original image corresponding to these 1 pixels (e.g., using the function 'bwselect' in MATLAB).
Sample segmented character from the number plate which is to be recognized Can be represented as f(x, y)
29
FIGURE 3.1 Segmented Character of the number plate
Template Image which is required to be recognized from number plate.
Can be represented as h(x, y)
FIGURE 3.2 Template Image
Result of Correlation of f(x, y) and h(x, y). G(x, y) = f(x, y) * (h*(x, y)).
FIGURE 3.3 Result Of Correlation
30
3.9 CONCLUSION The important techniques like signature, correlation and template matching are explained. Signature technique is used for locate the high frequency areas in the image. Correlation technique is used for identification of segmented characters from the number plate.
CHAPTER IV WORKING DESCRIPTION 4.1 INTRODUCTION This chapter deals with algorithm which we have implemented for extraction of characters from the image containing car number plate. Flowchart depicting the flow of processes involving in the project and the function flow diagrams, and code with neat comments.
4.2 ALGORITHM 31
This system takes a vehicle image of any size.
Breaks it into smaller image pieces.
These pieces are then analyzed to locate the exact location of number plate in the image.
Once the area of the number plate (its x and y coordinates) is found the plate is parsed to extract the character from it.
These characters are then recognized and converted to ASCII text format.
32
4.3 FLOWCHART DEPICTING THE OPERATIONS OF THE SYSTEM
START
Take a vehicle image of any size
Break image into smaller segments
Analyzing pieces to locate the number plate
Parsing the number plate to extract characters
Recognition of characters
Converting the recognized characters to text format
STOP FIGURE 4.1
4.4 FLOW OF FUNCTION CALLS IN THE PROGRAM 33
This flowchart shows the flow of function calls to the m files in the program starting with the main function ‘Tmain’
Tmain
LoadBmp
ImBin
NoBlack
CntPk
Plate
CntPkperRow
CntPk
ImBin
CntPk
NoBlack
CntPkperRow
CntPk
FIGURE 4.2
4.5 MATLAB CODE Tmain.m
34
OCR
Template
function Tmain() % Description about the function: % Main function of the program. clc;
% Clears matlab command window
close all;
% Closes figure windows
file = 'image2.bmp';
% Input image
x = LoadBmp(file);
% Calling "LoadBmp" function
figure,colormap(gray(256)),image(x);
% Displays input image in gray scale
y = ImBin(x,0);
% Calling ImBin function
Sel = NoBlack(y,0);
% Calling NoBlack function
noPlate = Plate(x,Sel,y);
% Calling Plate function
LoadBmp.m
function x = LoadBmp(fileName) %______________________________________________________________________ % Description about the function LoadBmp: % Loads the input image, converts it into double data type. %______________________________________________________________________ x= imread(fileName,'bmp');
% Reads image from the file specified by the string %"fileName", which is of the format "bmp".
x = double(x);
% Converts the datatype of x to double precision.
ImBin.m
function y=ImBin(x,InvFlg) %______________________________________________________________________ 35
% Descrption about the function ImBin: % This functions coverts the image stored in "x" to binary image (black and white) %______________________________________________________________________ temp = median(x);
% Finds the median value of the pixcels in each column of "x" % and returns it in a row vector ("temp")
med = median(temp);
% Finds median of the row vector temp
y = gt(x,med);
% Binary image is obtained
h = imhist(y); sz = size(h); "sz"
% Returns the sizes of each dimension of array "h" in a vector
if InvFlg == 1 if(h(end) < h(1)) y = 1 - y;
% Inverts the image
end else y = 1 - y;
% Inverts the image
figure,imshow(y);
% Displays the binarised image in a window
end NoBlack.m
function pkVlyCnt = NoBlack(img,CharFlg); %______________________________________________________________________ % Descrption about the function NoBlack: % This function takes binarised image as input and it returns the high % frequency areas of the image (one of them will be a number plate) %______________________________________________________________________ pkVlyCnt = zeros(1,4); 36
siz = size(img); if(siz(2) >2) rowHist = sum(img.');
% Signature for white pixels
rowHist = siz(2) - rowHist;
% Signature for black pixels
rowmin2 = min(rowHist); rowmax2 = median(rowHist);
binX = gt(rowHist,(rowmax2 + rowmin2)/2);
% Keeping the sensativity.
pkValy = CntPk(binX);
% Calling the function CntPk
pkVlyCnt = CntPkperRow(img, pkValy(2:end),CharFlg); % Calling the function CntPkperRow else pkVlyCnt = 0; end CntPk.m
function cnt = CntPk(x) %______________________________________________________________________ % Descrpition about the function CntPk % This function returns the co-ordinates of the image which corresponds to % high frequency areas and also the number of high frequency ares %______________________________________________________________________ sz = size(x); if(sz(2) > 2) index = 1; st = 1; nd = sz(2); 37
prev = x(1); cnt = prev; for i = 2 : sz(2) cur = x(i); if((prev == 0) & (cur == 1)) % Enters in the beginning of the peak (high frequency) st = i;
% Reads the corresponding co-ordinate
end if((prev == 1) & (cur == 0)) % Enters in the end of the peak (high frequency) cnt = cnt + 1;
% Updates the count
rig(index) = st;
% Stores the starting co-ordinate of the peak
rig(index + 1) = i;
% Stores the ending co-ordinate of the peak
index = index + 2; end prev = cur; end if(( i == sz(2)-1)&(st > rig(index-1))) cnt = cnt + 1; rig(index) = st; rig(index + 1) = i; index = index + 2; end if(cnt > 1) cnt = cnt - 1; cnt = [cnt rig];
% Places the number of peaks in the first position of array("cnt") % and remaining positions with co-ordinates of the image
end 38
else cnt = 0; end CntPkperRow.m
function Sel=cntPkperRow(img, x,CharFlg); %______________________________________________________________________% Description about the function CntPkperRow: % Depending on CharFlg % This function senses the high frequency areas (column wise)in the co-ordinats given % by CntPk.and gives them back to CntPk to locate the co-ordinates of them (OR) % Once the number plate is found this gives the segmented characters of it. %______________________________________________________________________ index = 1; Sel = zeros(1,4); saveIndex = 1; isz = size(img); sz = size(x); i=1; while(i < sz(2)) cnt1 = x(i); cnt2 = x(i+1); if(cnt2 - cnt1 > 3) colHist = sum(img(cnt1:cnt2,:)); pixels(column)
% Signature for white
colHist = isz(1) - colHist; pixels(column)
% Signature for black
39
if(CharFlg == 0) colmin2 = min(colHist); colmax2 = median(colHist); binY = gt(colHist, (colmax2 + colmin2)/2); % Sensing the high frequency areas else binY = gt(colHist, min(colHist)+1); % Sensing the high frequency ares (once the number plate is found) end
%'cntPkperRow while next ->' temp = cntPk(binY); % Calling the function CntPk to find the co-ordinates of the high frequency areas pkVlyCnt(index) = temp(1); index = index + 1; sztemp = size(temp); for ind=1:temp(1) if(sztemp(2) > 1) st = 2 * ind; en = (2 * ind) + 1; if((temp(en) - temp(st)) > 5) Sel(saveIndex,1:4) = [cnt1 cnt2 temp(st) temp(en)]; % Storing the co-ordinates of row and column high frequency ares saveIndex = saveIndex + 1; end end 40
st = 0; en = 0; end end i = i + 2; end Plate.m
function noPlate = Plate(img,Sel,image) %______________________________________________________________________ % Description about the function Plate: %This function takes the high frequency ares of the image as inpput and %finds the probable number plate and also extracts the charaters from it. %these characters are then given to "OCR" module for identification %______________________________________________________________________ szSel = size(Sel); selPlate = zeros(2,4); szPlate = size(selPlate); figure; for index = 1:szSel(1) subplot(szSel(1),1,index),imshow(image(Sel(index,1):Sel(index,2),Sel(index,3):Sel(index, 4))); % Displays the high frquency ares of the image end for i=1:szSel(1) P =imBin(img((Sel(i,1)):(Sel(i,2)),(Sel(i,3)):(Sel(i,4))),1); charSel = noBlack(P,1); 41
szTemp = size(charSel); if((szTemp(1)>1)&(szTemp(1) > szPlate(1))) selPlate = charSel; the image
%co-ordinates of the extracted characters from
binImg = P; end end figure,imshow(binImg);
%displays the probable number plate from the image
figure; szSel = size(selPlate); if(szSel(1) > 1) for ind = 1:szSel(1) subplot(szSel(1),1,ind),imshow(binImg(selPlate(ind,1):selPlate(ind,2),selPlate(ind,3): selPlate(ind,4))); match = OCR(binImg(selPlate(ind,1):selPlate(ind,2),selPlate(ind,3):selPlate(ind,4))); % Calling "OCR" function (passes the segmented characters to the OCR) switch match case 1 lnum(ind) = '3'; case 2 lnum(ind) = '5'; case 3 lnum(ind) = '6'; case 4 lnum(ind) = 'G'; case 5 lnum(ind) = 'H'; 42
case 6 lnum(ind) = 'K'; case 7 lnum(ind) = 'W'; case 8 lnum(ind) = '8'; case 9 lnum(ind) = '1'; case 10 lnum(ind) = 'A'; case 11 lnum(ind) = 'D'; end end end fid = fopen('text.txt', 'wt'); fprintf(fid,'%s\n',lnum); % Displays characters of number plate in ASCII format in a text file fclose(fid); winopen('text.txt') noPlate = selPlate; OCR.m
function tempInd = OCR(img) %______________________________________________________________________ % Desription about the function OCR: % This function sends the segmented characters from the number plate and the templates 43
% to "Template" function, which returns the correlated results.From these results this % finds the best match and passes the template number to "Plate" function. %______________________________________________________________________ filename = ['3.bmp';'5.bmp';'6.bmp';'g.bmp';'h.bmp';'k.bmp';'w.bmp';'8.bmp';'1.bmp';'A.bmp';'D.bmp' ]; % templates from database szf = size(filename); tempInd = 0; for i = 1:szf(1) temp = loadBmp(filename(i,:));
% Reads the templates from "filename"
num = template(img,temp); % Calling function "Template".Pases the segmented image and a template for matching dstort(i) = num;
% Stores the correlated results
end match = 1; sz = size(dstort); match = max(dstort);
% Stores the maximum correlated value
for i=1:sz(2) if match == dstort(i) tempInd = i; % stores the template number which has the maximum correlated value end end Template.m
function maximum=template(bw, a); %______________________________________________________________________% Description about the function Template: 44
% This program inputs two images. % 1) Segmented Image on which a character is required to be recognized % 2) Image of character template which is required to be recognized on % the given image. % This outputs the correlated results to "template" function %______________________________________________________________________ bw=bwmorph(bw,'clean');
%performs removal of isolated pixels from image.
bw=bwmorph(bw,'spur');
%performs removal of end points from image.
a=bwmorph(a,'clean');
%performs removal of isolated pixels from image.
a=bwmorph(a,'spur');
%performs removal of end points from image.
bw=imresize(bw,[42 32],'nearest'); % returns an image bw that is of size [42 32], using nearest-neighbor interpolation. a=imresize(a,[42 32],'nearest'); selected = bw; result=real(ifft2(fft2(selected,70,324) .* fft2(rot90(a,2),70,324))); %correlation of the segmented image and the template maximum=max(result(:)); % Stores the maximum result obtained from correlation.
4.6 GUI FIGURE CODE function varargout = Project_gui(varargin) % PROJECT_GUI M-file for Project_gui.fig %
PROJECT_GUI, by itself, creates a new PROJECT_GUI or raises the existing
%
singleton*.
%
H = PROJECT_GUI returns the handle to a new PROJECT_GUI or the handle to
%
the existing singleton*.
%
PROJECT_GUI('CALLBACK',hObject,eventData,handles,...) calls the local 45
%
function named CALLBACK in PROJECT_GUI.M with the given input arguments
%
PROJECT_GUI('Property','Value',...) creates a new PROJECT_GUI or raises the
%
existing singleton*. Starting from the left, property value pairs are
%
applied to the GUI before Project_gui_OpeningFunction gets called. An
%
unrecognized property name or invalid value makes property application
%
stop. All inputs are passed to Project_gui_OpeningFcn via varargin.
%
GUI Options on GUIDE's Tools menu. Choose "GUI allows only one
%
instance to run (singleton)".
% Edit the above text to modify the response to help Project_gui % Last Modified by GUIDE v2.5 07-Apr-2009 19:59:37 % Begin initialization code - DO NOT EDIT gui_Singleton = 1; gui_State = struct('gui_Name',
mfilename, ...
'gui_Singleton', gui_Singleton, ... 'gui_OpeningFcn', @Project_gui_OpeningFcn, ... 'gui_OutputFcn', @Project_gui_OutputFcn, ... 'gui_LayoutFcn', [] , ... 'gui_Callback', []); if nargin && ischar(varargin{1}) gui_State.gui_Callback = str2func(varargin{1}); end if nargout [varargout{1:nargout}] = gui_mainfcn(gui_State, varargin{:}); else gui_mainfcn(gui_State, varargin{:}); end 46
% End initialization code - DO NOT EDIT % --- Executes just before Project_gui is made visible. function Project_gui_OpeningFcn(hObject, eventdata, handles, varargin) % This function has no output args, see OutputFcn. % hObject
handle to figure
% eventdata reserved - to be defined in a future version of MATLAB % handles
structure with handles and user data (see GUIDATA)
% varargin command line arguments to Project_gui (see VARARGIN) % Choose default command line output for Project_gui handles.output = hObject; % Update handles structure guidata(hObject, handles); % UIWAIT makes Project_gui wait for user response (see UIRESUME) % uiwait(handles.figure1); % --- Outputs from this function are returned to the command line. function varargout = Project_gui_OutputFcn(hObject, eventdata, handles) % varargout cell array for returning output args (see VARARGOUT); % hObject
handle to figure
% eventdata reserved - to be defined in a future version of MATLAB % handles
structure with handles and user data (see GUIDATA)
% Get default command line output from handles structure varargout{1} = handles.output; % --- Executes on selection change in popupmenu1. function popupmenu1_Callback(hObject, eventdata, handles) global file; contents = get(hObject,'String'); 47
a=contents{get(hObject,'Value')} ; if(a=='Image1') file=('Image1.bmp'); end if(a=='Image2') file=('Image2.bmp'); end % hObject
handle to popupmenu1 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB % handles
structure with handles and user data (see GUIDATA)
% Hints: contents = get(hObject,'String') returns popupmenu1 contents as cell array %
contents{get(hObject,'Value')} returns selected item from popupmenu1
% --- Executes during object creation, after setting all properties. function popupmenu1_CreateFcn(hObject, eventdata, handles) % hObject
handle to popupmenu1 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB % handles
empty - handles not created until after all CreateFcns called
% Hint: popupmenu controls usually have a white background on Windows. %
See ISPC and COMPUTER.
if ispc set(hObject,'BackgroundColor','white'); else set(hObject,'BackgroundColor',get(0,'defaultUicontrolBackgroundColor')); end % --- Executes on button press in pushbutton1. function pushbutton1_Callback(hObject, eventdata, handles) 48
global file; Tmain(file,1); % hObject
handle to pushbutton1 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB % handles
structure with handles and user data (see GUIDATA)
% --- Executes on button press in pushbutton2. function pushbutton2_Callback(hObject, eventdata, handles) global file; Tmain(file,2); % hObject
handle to pushbutton2 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB % handles
structure with handles and user data (see GUIDATA)
% --- Executes on button press in pushbutton3. function pushbutton3_Callback(hObject, eventdata, handles) global file; Tmain(file,3); % hObject
handle to pushbutton3 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB % handles
structure with handles and user data (see GUIDATA)
% --- Executes on button press in pushbutton4. function pushbutton4_Callback(hObject, eventdata, handles) global file; Tmain(file,4); % hObject
handle to pushbutton4 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB % handles
structure with handles and user data (see GUIDATA) 49
% --- Executes on button press in pushbutton5. function pushbutton5_Callback(hObject, eventdata, handles) global file; Tmain(file,5); % hObject
handle to pushbutton5 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB % handles
structure with handles and user data (see GUIDATA)
% --- Executes on button press in pushbutton6. function pushbutton6_Callback(hObject, eventdata, handles) global file; Tmain(file,6); % hObject
handle to pushbutton6 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB % handles
structure with handles and user data (see GUIDATA)
% --- Executes on button press in pushbutton7. function pushbutton7_Callback(hObject, eventdata, handles) global file; Tmain(file,7); % hObject
handle to pushbutton7 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB % handles
structure with handles and user data
4.7 CONCLUSION
This chapter discussed flow of function calls in the MATLAB code. MATLAB Code and GUI code with comments is also given. Tmain is the main function. ImBin
50
converts the image to binary. NoBlack gives the coordinates of the number plate in the image. Plate gives the final output of characters of the number plate in ASCII format.
CHAPTER V ADVANTAGES, DISADVANTAGES & APPLICATIONS 5.1 ADVANTAGES
Simple and easy to implement
Good performance
Translation-invariant The CNPR system significant advantage is that the system can keep an image
record of the vehicle which is useful in order to fight crime and fraud ("an image is worth a thousand words"). An additional camera can focus on the driver face and save the image for security reasons. Additionally, this technology does not need any installation per car (such as in all the other technologies that require a transmitter added on each car or carried by the driver).
5.2 DISADVANTAGES
Requires considerable tweaking to find the right templates Not scale or rotation invariant. 51
Performance deteriorates rapidly for incomplete or noisy data
5.3 APPLICATIONS •
Traffic Congestion Management (differential pricing for use of roads in a precinct)
•
Traffic Speed Restrictions
•
Access Control.
•
Tolling
•
Border Control
•
Offender Identification (unregistered, uninsured, stolen vehicles, fake number plates)
•
Criminal Intelligence (location of terrorism suspects and 'persons of interest to the police')
•
Generation of blacklists of vehicles seeking fuel or other products from petrol stations
5.3.1 Traffic Congestion
Traffic congestion schemes, of increasing significance in urban centres where there are concerns about pollution or the capacity of streets to accommodate a large number of vehicles during peak hours, seek to exclude classes of motor vehicles (for example cars 52
owned by non-residents) or to allow entry of vehicles on a selective basis (e.g. at a particular time or subject to a financial penalty). In essence most schemes involve charging for use of public streets and have thus proved to be controversial, although sometimes supported by residents of the particular precinct. This system has been used in such schemes to identify what vehicles are traversing a location, on the basis that the technology is less "invasive" than electronic tagging systems that require users to gain a RFID tag for the particular driver or vehicle. The registration database is typically linked to a billing system, with this system being used to identify 'authorized' vehicles (which have usually paid for the privilege of using the local road network) and non-authorized vehicles, which are automatically issued with a fine. Given the speed of processing and sufficient cameras it is possible to develop highly granular, time-specific and dynamic charging, e.g. different charges at various times of the day, higher charges when pollution reaches particular levels. 5.3.2 Speed Restrictions
This system can also be used for automated enforcement of speed restrictions and associated traffic codes, such as prohibitions on particular classes of vehicles (such as those carrying explosives or chemical) using specific routes. Speed restriction is mechanistic, with the this system 'logging' the vehicle at two or more locations and determining whether transit between those points and times breached speed limits. A billing database can then automatically issue a fine and/or alert traffic police that a breach is underway. 5.3.3 Access Control
A gate automatically opens for authorized members in a secured area, thus replacing or assisting the security guard. The events are logged on a database and could be used to search the history of events. 5.3.4 Tolling
53
The car number is used to calculate the travel fee in a toll-road, or used to doublecheck the ticket. In this installation, the plate is read when the vehicle enters the toll lane and presents a pass card. The information of the vehicle is retrieved from the database and compared against the pass information. In case of fraud the operator is notified. 5.3.5 Network Management
This is academic and official interest, although apparently few implementations, in use of this system to identify how road networks are being used and thereby better model future traffic flows or underpin intelligent transport systems (e.g. dynamic signage and traffic signals that reflect loads on parts of the network). 5.3.6 Offender Identification
Law enforcement officials appear to have been more enthusiastic about the scope for using this system in ‘offender identification’. That encompasses such things as identification of •
unregistered vehicles, i.e. vehicles with an expired number plate (significant because most regimes prohibit driving of an unregistered vehicle but do not require that 'unused' vehicles have a current plate)
•
uninsured vehicles (primarily in regimes where it is mandatory to insure a vehicle through a government-controlled insurer)
•
stolen vehicles (i.e. where the thief takes the vehicle but does not change the number plates)
•
vehicles with cloned plates (e.g. the same plates are identified on separate vehicles in different locations at much the same time) or with bogus numbers (e.g. there is no match with the vehicle register)
•
Vehicles engaged in activity such as unauthorized use of bus-only lanes. Such identification can be tied to automated issue of a fine or summons to appear
before a magistrate. It can also be used in real time to alert law enforcement personnel to
54
intercept the suspect vehicle, for example to find and pull over a car that has been reported as stolen. 5.3.7 Criminal Intelligence
Police, customs and other law enforcement officials are reluctant to tightly circumscribe use of this system. As the following pages note there appears to be considerable sharing of information by agencies and real time checking of plates against multiple hotlists.
Such lists include vehicles – •
Reported to have been involved in hit and run incidents, petrol thefts, drive by shootings, burglaries and abductions
•
Reported as having been at the scene of a crime
•
Registered to suspected terrorists, family and associates
•
Rented using 'flagged' credit cards
•
Registered to suspected drug traffickers
•
Registered to suspected stalkers In practice a wide range of people may be 'of interest to the police' (and their
peers); the major limit on real-time large scale ad hoc sorting based on this system is the difficulty of linking different databases. Blacklists and Whitelists
The preceding applications have primarily involved official ANPR (Automatic Number Plate Recognition) systems, particularly those tied to one or more government databases. It is important, however, to recognize that private ANPR schemes have been established in many countries.
55
One Australian promoter for example has marketed a commercial ANPR network to petrol stations, with individual stations capturing an image of the plate on each vehicle entering those premises. The expectation is that if there is a 'drive off' (i.e. the driver leaves the station without paying) the site manager will manually apply a 'flag' to the corresponding license number on the station's database. If the vehicle returns to the site an alarm will be sounded by the system and the cashier will disable the pump. That blacklist can be shared with other stations via periodic exchange of information over the promoter's server, a "feature that prevents an offending vehicle stealing fuel from a different site every time". Other vendors are promoting use of ANPR as an access control mechanism for entry to a building or campus, with the vehicle's number plate replacing a proximity card or other identifier.
CHAPTER VI RESULTS & CONCLUSIONS This chapter deals with the results obtained at the various stages of the project. And the final output of the project in ASCII format.
6.1 RESULTS STEP 1
Photograph of the car which contains number plate portion taken by CCTV camera.
56
FIGURE 6.1 Input image to Tmain
Size:
254x333
Bytes: Class:
uint8 array
STEP 2
Binarised form of the above image, output of Imbin function. (If the image is binarised then most of the detail is lost from the image, leaving our area of interest more prominent.)
57
FIGURE 6.2 Binarised image of input image
STEP 3
High frequency areas of the image to find the probable location of the number plate (Using row wise histogram)
58
FIGURE 6.3 Row wise segmentation of the Binarised Iimage
STEP 4
After column wise histogram (for the above obtained result)
59
FIGURE 6.4 High frequency areas of the image (after column wise segmentation)
STEP 5 Number plate
60
FIGURE 6.5 Number Plate from the Input Image ( Binarised Image)
STEP 6
Segmented characters from the above obtained number plate.
61
FIGURE 6.6 Segmented characters of the number plate
OUTPUT
Output of the system in ASCII format GHW356K
6.2 CONCLUSION
62
As described in the project, this system provides characters of the car number plate in ASCII format that can be used for various applications. This is a comprehensive step providing traffic management, security, control and offender identification. This technology is simple and does not need any installations on cars.
6.3 FUTURE ADVANCEMENTS
The software can be used along with camera to serve many applications and we can even make the system automatic
An additional camera can focus on the driver face and save the image for security reasons. In order to reduce the execution time of the code this can be implemented using c/c++.
APPENDIX A 63
IMAGE PROCESSING TOOLBOX INTRODUCTION The Image Processing Toolbox is a collection of functions that extend the capability of the MATLAB numeric computing environment. The toolbox supports a wide range of image processing operations, including
Spatial image transformations
Morphological operations
Neighborhood and block operations
Linear filtering and filter design
Transforms Image analysis and enhancement
Image registration
Deblurring
Region of interest operations Many of the toolbox functions are MATLAB M-files, a series of MATLAB
statements that implement specialized image processing algorithms. The capabilities of the Image Processing Toolbox can be extended by writing user defined M-files, or by using the toolbox in combination with other toolboxes, such as the Signal Processing Toolbox and the Wavelet Toolbox
IMAGES IN MATLAB The basic data structure in MATLAB is the array, an ordered set of real or complex elements. This object is naturally suited to the representation of images, real-valued ordered sets of color or intensity data. MATLAB stores most images as two-dimensional arrays (i.e., matrices), in which each element of the matrix corresponds to a single pixel in the displayed image. (Pixel is derived from picture element and usually denotes a single dot 64
on a computer display.) For example, an image composed of 200 rows and 300 columns of different colored dots would be stored in MATLAB as a 200-by-300 matrix. Some images, such as RGB, require a three-dimensional array, where the first plane in the third dimension represents the red pixel intensities, the second plane represents the green pixel intensities, and the third plane represents the blue pixel intensities. This convention makes working with images in MATLAB similar to working with any other type of matrix data, and makes the full power of MATLAB available for image processing applications. For example, we can select a single pixel from an image matrix using normal matrix subscripting. I (2, 15) This command returns the value of the pixel at row 2, column 15 of the image I.
COORDINATE SYSTEMS Locations in an image can be expressed in various coordinate systems, depending on context. This section discusses the two main coordinate systems used in the Image Processing Toolbox and the relationship between them. These two coordinate systems are described in
Pixel Coordinates
Spatial Coordinates
Pixel Coordinates
Generally, the most convenient method for expressing locations in an image is to use pixel coordinates. In this coordinate system, the image is treated as a grid of discrete elements, ordered from
top to bottom and
left
illustrated by the
to
right,
as
following figure.
65
FIGURE. A.1 Pixel Coordinate System
For pixel coordinates, the first component r (the row) increases downward, while the second component c (the column) increases to the right. Pixel coordinates are integer values and range between 1 and the length of the row or column. There is a one-toone correspondence between pixel coordinates and the coordinates MATLAB uses for matrix subscripting. This correspondence makes the relationship between an image's data matrix and the way the image is displayed easy to understand. For example, the data for the pixel in the fifth row, second column is stored in the matrix element (5, 2). Spatial Coordinates
In the pixel coordinate system, a pixel is treated as a discrete unit, uniquely identified by a single coordinate pair, such as (5, 2). From this perspective, a location such as (5.3,2.2) is not meaningful. At times, however, it is useful to think of a pixel as a square patch. From this perspective, a location such as (5.3, 2.2) is meaningful, and is distinct from (5, 2). In this spatial coordinate system, locations in an image are positions on a plane, and they are described in terms of x and y (not r and c as in the pixel coordinate system). The following figure illustrates the spatial coordinate system used for images. Notice that y increases
66
FIGURE A.2 Spatial Coordinate System
Differences between Pixels Coordinate System and Spatial Coordinate System
There are some important differences, however. In pixel coordinates, the upper left corner of an image is (1, 1), while in spatial coordinates, this location by default is (0.5, 0.5). This difference is due to the pixel coordinate system's being discrete, while the spatial coordinate system is continuous. Also, the upper left corner is always (1, 1) in pixel coordinates, but you can specify a nondefault origin for the spatial coordinate system. See Using a Nondefault Spatial Coordinate System for more information.
STORAGE CLASSES SUPPORTED BY MATLAB AND IPT Images are most commonly stored in MATLAB using the logical, uint8, uint16 and double data types. By default, MATLAB stores most data in arrays of class double. The data in these arrays is stored as double-precision (64-bit) floating-point numbers. All MATLAB functions work with these arrays. For image processing, however, this data representation is not always ideal. The number of pixels in an image can be very large. To reduce memory requirements, MATLAB supports storing image data in arrays as 8-bit or 16-bit unsigned integers, class 67
uint8 and uint16. These arrays require one eighth as much memory as double arrays and can perform many standard MATLAB array manipulations. The data classes listed in the below table are supported by most IPT functions. Data Class Double
Description Double-precision, floating point numbers in the approximate range -10^308 to 10 ^308 (8 bytes per element).
Unit8
Unsigned 8-bit integers in the range [0,255] (1 byte per element).
Unit16
Unsigned 16-bit integers in the range [0, 65535] (2 bytes per element).
Unit32
Unsigned 32-bit integers in the range [0, 4294967295] (4 bytes per element).
Int8
Signed 8-bit integers in the range [-128, 127] (1 byte per element).
Int16
Signed 16-bit integers in the range [-32768, 32767] (2 bytes per element).
Int32
Signed 32-bit integers in the range [-2147483648, 2147483647] (4 bytes per element).
Single
Single-precision, floating point numbers in the approximate range -10^38 to 10 ^38 (4 bytes per element).
Char
Characters (2 bytes per element)
Logical
Values are 0 or 1 (1 byte per element)
TABLE A.1 Data Classes supported by IPT functions
Certain MATLAB functions, including the find, all, any, conv2, convn, fft2, fftn, and sum functions, accept uint8 or uint16 data but return data in double-precision format.
TYPES OF DIGITAL IMAGES 68
The Image Processing Toolbox supports four basic types of images:
Binary Images
Intensity Images
Indexed Images
RGB (True Colour) Images
Binary Images
In a binary image, each pixel assumes one of only two discrete values. Essentially, these two values correspond to on and off. A binary image is stored as a logical array of 0's (off pixels) and 1's (on pixels).
FIGURE A.3 Binary Image
Intensity Images
An intensity image is a data matrix, I, whose values represent intensities within some range. MATLAB stores an intensity image as a single matrix, with each element of 69
the matrix corresponding to one image pixel. The matrix can be of class double, uint8, or uint16.While intensity images are rarely saved with a colormap, MATLAB uses a colormap to display them. The elements in the intensity matrix represent various intensities, or gray levels, where the intensity 0 usually represents black and the intensity 1, 255, or 65535 usually represents full intensity, or white.
FIGURE A.4 Intensity Image
Indexed Image
An indexed image consists of a data matrix, X, and a colormap matrix, map. The data matrix can be of class uint8, uint16, or double. The colormap matrix is an m-by-3 array of class double containing floating-point values in the range [0,1]. Each row of map specifies the red, green, and blue components of a single color. An indexed image uses direct mapping of pixel values to colormap values. The color of each image pixel is determined by using the corresponding value of X as an index into map. The value 1 points to the first row in map, the value 2 points to the second row, and so on.
70
FIGURE A.5 Indexed Image
RGB (True Colour) images
An RGB image, sometimes referred to as a true-color image, is stored in MATLAB as an m-by-n-by-3 data array that defines red, green, and blue color components for each individual pixel. RGB images do not use a palette. The color of each pixel is determined by the combination of the red, green, and blue intensities stored in each color plane at the pixel's location. Graphics file formats store RGB images as 24-bit images, where the red, green, and blue components are 8 bits each. This yields a potential of 16 million colors. The precision with which a real-life image can be replicated has led to the commonly used term true-color image. An RGB array can be of class double, uint8, or uint16. In an RGB array of class double, each color component is a value between 0 and 1. A pixel whose color components are (0, 0, 0) is displayed as black, and a pixel whose color components are (1, 1, 1) is displayed as white. The three color components for each pixel are stored along the third dimension of the data array. For example, the red, green, and blue color components of the pixel (10,5) are stored in RGB(10,5,1), RGB(10,5,2), and RGB(10,5,3), respectively.
71
FIGURE A.6 RGB (True Colour) Image
Multiframe image arrays
For some applications, you might need to work with collections of images related by time or view, such as magnetic magnetic resonance resonance imaging imaging (MRI) (MRI) slices or movie frames. The Image Processing Toolbox provides support for storing multiple images in the same array. Each Each sepa separa rate te image image is calle called d a frame frame.. If an arra array y holds holds multi multipl plee fram frames es,, they they are are concatenated along the fourth dimension. For example, an array with five 400-by-300 RGB images would be 400-by-300-by-3-by-5. A similar multiframe intensity or indexed image would be 400-by-300-by-1-by-5. Use the cat command to store separate images in one multiframe array. For example, if you have a group of images A1, A2, A3, A4, and A5, you can store them in a single array using A = cat(4,A1,A2,A3,A4,A5) We can also extract frames from a multiframe image. For example, if you have a multiframe image MULTI, this command extracts the third frame. FRM3 = MULTI (:,:,:, 3)
72
Note that, in a multiframe multiframe image image array, each image must must be the same size and have the same number of planes. In a multiframe indexed image, each image must be the same size and have the same number of planes. In a multiframe indexed image, each image must also use the same colormap.
SUMMARY OF IMAGE TYPES AND DATA CLASSES
Image Type
Data Class
Interpretation
Binary
logical
Array of zeros (0) and ones (1)
Indexed
double
Array of integers in the range [1, p]. The asso associa ciate ted d color colorma map p is a p-by p-by-3 -3 arra array y of floating-point values in the range [0, 1]
uint uint8 8 or uint uint1 16
Array rray of int integer egerss in the the rang range [0, [0, p-1] p-1].. The asso associa ciate ted d color colorma map p is a p-by p-by-3 -3 arra array y of floating-point values in the range [0, 1].
double
Array of of fl floating-point values. Th The ty typical rang rangee of value valuess is [0, [0, 1]. The The asso associa ciate ted d colorm colormap, ap, typical typically ly graysca grayscale, le, is a p-by-3 p-by-3 array of floating-point values in the range [0, 1].
uint uint8 8 or uint uint16 16
Arra Array y of int integ eger ers. s. The The typ typic ical al ran range ge of of valu values es is [0, [0, 255] 255] or [0, [0, 65535 65535]. ]. The The asso associa ciate ted d colorm colormap, ap, typical typically ly graysca grayscale, le, is a p-by-3 p-by-3 array of floating-point values in the range [0, 1].
Intensity
RGB color)
(true- double
uint uint8 8 or uint uint16 16
m-by-n-by-3 array of floating-point values in the range [0, 1] m-by m-by-n -n-b -byy-3 3 arra array y of int integ eger erss in the the ran range ge [0, [0, 255] or [0, 65535]
TABLE A.2 Image Types and Numeric Classes
73
IMAGE TYPE CONVERSION FUNCTIONS For certain operations, it is helpful to convert an image to a different image type
Function
Description
dither
Create a bi binary im image fr from a grayscale in intensity image by dithering; create an indexed image from an RGB image by dithering
gray2ind
Create an an indexed im image from a grayscale in intensity image
grayslice
Create an an in indexed im image fr from a grayscale in intensity image by thresholding
im2bw
Create a binary im image fr from an an in intensity im image, inde ndexed image, or RGB image, based on a luminance threshold
ind2gray
Create a grayscale intensity image from an in indexed image
ind2rgb
Create an RGB image from an indexed image
mat2gray
Create a gr grayscale in intensity im image fr from da data in in a matrix, by scaling the data
74
TABLE A.3 Image Type conversion Functions
IMAGE ARITHMETIC Image arithmetic is the implementation of standard arithmetic operations, such as addition, subtraction, multiplication, and division, on images. Image arithmetic has many uses in image processing both as a preliminary step in more complex operations and by itself. For example, image subtraction can be used to detect differences between two or more images of the same scene or object. we can do image arithmetic using the MATLAB arithmetic operators. The Image Processing Toolbox also includes a set of functions that implem implement ent arithm arithmetic etic operatio operations ns for all numeric numeric,, nonspar nonsparse se data types. types. The toolbox toolbox arithmetic functions accept any numeric data type, including uint8, uint16, and double, and return the result image in the same format. The functions perform the operations in double precision, precision, on an element-by-ele element-by-element ment basis, but do not convert images to double-precis double-precision ion
75
values in the MATLAB workspace. Overflow is handled automatically. The functions truncate return values to fit the data type. Image Arithmetic Truncation Rules
The results of integer arithmetic can easily overflow the data type allotted for storage. For example, the maximum value you can store in uint8 data is 255. Arithmetic operations can also result in fractional values, which cannot be represented using integer arrays. The image arithmetic functions use these rules for integer arithmetic: Values that exceed the range of the integer type are truncated to that range. Fractional values are rounded. For example, if the data type is uint8, results greater than 255 (including Inf) are set to 255.
BIBLIOGRAPHY 1.
Automatic Number Plate Recognition – Universty of Trás-os-Montes e Alto Douro - Vila Real – Portugal.
2.
CARINA - Software Product for Automatic Number Plate Recognition – Budapest – Hungary.
3.
Real-time number plate reading , J. Bulas-Cruz , J. Barroso, A Rafaele E. L. dagless
4.
http://www.mathworks.com/
5.
Digital Image Processing Using MATLAB 2nd Edition By Rafael C. Gonzalez, Richard E. Woods and Steven L. Eddins
76