LabVIEW Machine Vision for Line Detection EGR315: Instrumentation Spring 2010 Than Aung and Mike Conlow Department of Physics and Engineering Elizabethtown College Elizabethtown, Pennsylvania Email:
[email protected] [email protected],, and
[email protected] [email protected].. Abstract
– To improve the previous
development avoidance proprietary
of
a
algorithm software
visual
obstacle
without that
using is
not
financially practical to purchase.
The
goal is to use more advanced methods to produce more concise output in terms of turning angle and the nearest point of interest.
package a prototype virtual instrument was developed to attempt to improve processing speed by using LabVIEW for the entire image processing procedure through a USB webcam [6]. There were several improvements that needed to be made to the prototype in order to justify its implementation over the previous
I. Introduction
The following data is an analysis of improvements made to a system previously developed using NI Vision Development software to detect white lines on non uniform grass. The need for this arose due to over complexity of a vision system that is on an autonomous robot that is used as a learning platform. The current system [5] uses a DVT Legend 554C that collects and filters the images internally and transmits the relevant data, via TCP/IP string, to a LabVIEW program that is a closed loop motor control. During the fall semester of 2009 using the NI Vision Development
vision
system.
The
turning
algorithm depended on a set of line detection
sub
virtual
instruments
that
generated large amounts of noise due to inadequate intensity filtering. To resolve these
and
other
thresholding,
and
programmed
using
issues line
the
filtering,
detection
base
package
were of
LabVIEW, using LabVIEW IMAQ and IMAQ USB to capture the images from a webcam [6]. The result is a great improvement over the previous version. Further enhancements still need to be implemented in order to operate properly in the field, but the goals that were set for this semester have been met.
calculated by averaging HL1 and HL2. The
II. Background
The previous project mainly employed NI Vision Development Module 9.0 (Trial
line angle is then calculated by find the angle between HCL and VCL, by using
Version), which provides various image processing and machine vision tools. By using
the
edge-detection
sub
virtual
instrument we implemented the following
Where m2 is the slope of HCL and m 1 is the slope of VCL. By using the intersection point and the angle between HCL and VCL,
line detection algorithm. The image resolution is set to 320x240 pixels capturing at 8 frames per second. Each frame is then converted to an 8 bit gray-scale image and then the image is
the appropriate heading for the robot is determined. Although the algorithm seems simple enough, there are a lot of drawbacks. First, when converting from 32 bit color images to
segmented into regions as follows [6]:
8 bit gray-scale images there is a loss of edge information in every frame. In the presence of background noises it is very difficult to detect stable edges, thereby making line detection less accurate. Second, using four edge detectors is unnecessarily redundant, and over-use of edge detectors results in slower processing. Third, we did not have time to implement the filters to Figure 1: Edge Detection Regions White lines are detected with IMAQ Edge Detection by finding lines in the eight boarder regions represented in green in Figure 1. In our algorithm, we use two vertical lines (VL1 and VL2) and two horizontal lines (HL1 and HL2) detectors. VCL
(Vertical
Center
Line)
is
then
calculated by averaging VL1 and VL2. Likewise, HCL (Horizontal Center Line) is
eliminate the noises and to threshold the unnecessary
pixels
information.
Finally,
since we used the 30-day trial version of NI Vision Development Module, to continue using the program the only option was to purchase the three thousand dollar full version. Therefore, the primary motivation of our project was to solve the problems we faced
by using NI Vision Development, and
amount of blue the most intense blues would
improve upon the shortcomings of the first
be whites.
project. With the main goals in mind, we
In
binary
format,
32
bits
color
is
developed the second version of our line
represented as follows:
detection algorithm.
Alpha
Red
Green
Blue
xxxx xxxx
xxxx xxxx
xxxx xxxx
xxxx xxxx
III. Implementation
Our project goals were to reduce the noises during the image acquisition, enhance
(x is a binary 1 or 0). In order to extract Blue
the edge information, and stabilize the
color information, we performed an AND
detected line even with the background
operation with the 32 color bits with the
reflections and light sources. Therefore, we
following binary bit mask [2].
divided the project into different modular
0000 0000
0000 0000
0000 0000
1111 1111
processes to achieve these goals. A. Single Color Extraction
The images acquired from the camera (Creative VF-0050) are 320x240 32-bit color images. Although we can simply convert the 32-bit color (RGB) images to 8bit grayscale images by averaging the 32-bit color, we have learned how to use a better method for eliminating the noises and
Figure 2: 32-bit Color Image
enhancing the edge information. Since the background of the images is mostly green, we decided that if we just simply extract the blue color pixels from RGB the images, we can reduce the noises and enhance the white color lines. The thought process behind this is that the dirt and grass are mostly composed of reds and greens, so if we were to only look at objects composed of some
Figure 3: 8-bit Blue Color Image
Figure 5: LabVIEW Convolution The figure above is the virtual instrument for a convolution, where X is the image matrix and Y is the convolution kernel. Since there are many convolutions being performed the algorithm is using frequency Figure 4: Blue Color Extraction
domain convolution. The way the operation
By extracting the blue color plane from an
is performed first requires that the image
image only pixels with a high blue intensity
matrix is padded horizontally and vertically
will appear white. This will reduce some
by one minus the width and height of the
noise from high intensity greens and reds.
kernel [4]. Then by shifting the kernel over
To eliminate noise from natural reflections a
the image the padded matrix is given the
spatial convolution using a flattening filter
values from the convolution, which is the
will be used to further enhance the image
Fourier Transform of X and Y multiplied
edges.
together as a dot product in terms of two
B. Spatial Convolution Filter
dimensional matrices and then the result is
To prevent large quantities of noise it was
transformed back to give the value desired
necessary to implement a convolution using
in the original resolution of 320x240 [4].
a 7x7 flatten kernel [2]. Since the image is
The following figure is a representation of
represented as a two dimensional matrix of
how special convolution is utilized to apply
blue
a kernel to a simple set of data.
intensity
values,
after
the
color
extractions, a convolution using a kernel of all ones will be applied to the image reducing high level intensity values. The reason this is done after the blue plane is extracted is to prevent the high intensity greens and reds from mixing with blues causing the blue plane to be inaccurate when extracted. Figure 6: Padding and Filtering
From the resulting matrix it is possible to
of each intensity value. Once we know the
get the average value of every 7x7 value by
intensity values of the image and their
dividing the elements by forty-nine to get
frequencies, it will become easier for us to
the values in the matrix in terms one byte
determine the edges that we are interested.
per index. Now all the high intensity noise that would have thrown off the later line detection should be gone as long as the noise doesn’t appear in large groups. From the flattened image it should now be much
easier
to
find
the
white
lines.
However, since the image was flattened the line edges will not be as intense as they were. So the next step is to determine the
Figure 7: Intensity Histogram
highest intensity values in the image to
In Figure 7, we can see clearly that the
attempt to only detect the highest intensities
maximum image intensity is around 200,
in a set range.
and minimum image intensity is around 15.
C. Intensity Analysis
However,
even
with
the
different
and
background lighting, there is one thing we
filtering, the image seems to be ready for
know for sure: the white lines always have
edge detection. However, there is still one
the maximum intensity. Therefore, if we can
problem we have to solve before performing
extract the intensity range from 180 to 200,
the edge detection. Under non-uniform
we can detect the white lines of the image.
After
the
single-color
extraction
background lighting, the maximum image intensity and intensity distribution of the image change accordingly, and it is almost impossible to perform normal thresholding to detect the edges. Therefore, we need to analyze the image intensity distribution. In order to do so, we first tried to acquire the intensity histogram of the image, which includes both intensity range and frequency
this case we will use a variable threshold value, which will be adjusted according to the background lighting as discussed in the previous section. Since we already know the maximum intensity of the image from the intensity analysis, then we will calculate the variable threshold as follows:
Figure 8: Intensity Analysis In Figure 8 it can be seen that the highest value found in the histogram is being passes on to the next part of the program. Also at the bottom of the figure there is a user
Figure 9: Adaptive Thresholding
control called Interval that contains the range of intensities accepted as white. This is where the adaptive threshold will receive its maximum and intensity range from. D. Adaptive Thresholding
Actually, thresholding is the simplest most powerful method for image segmentation. Mathematically,
thresholding
can
be
E. Hough Transformation
described as [1]:
Figure 10: Thresholded Image/ Interval = 20
Once we get the edge pixels after the adaptive thresholding we need to link them
, where f(x,y) is the input image, g(x,y) is
together to form a meaningful line. To
the threshold output image and T the
accomplish
threshold. Generally, thresholding uses the
Transformation will be used to bridge any
fixed value of T to segment the images. In
gaps in the line that may appear. This will
this
task
a
Hough
give us the position and direction of the line in the field of view [1] [2]. In Hough Transformations, each pixel (x, y) is transformed from Cartesian Space to (R,θ), as follows: Hough Space, H (R,θ),
, where 0 < R < and . If two pixels (x , y ) and (x , y ) are
Figure 11: Hough Space (R, θ)
co-linear, we will get the same value for R
Once we get the R and θ, we need to shift
and θ. In other words, a line in Cartesian
them as follows:
1
1
2
2
Space is represented as a point in Hough Space. A simple Hough Transformation can be achieved by using a two-dimensional
accumulator (array), which corresponds to R and θ. Each cell of the accumulator is defined by unique R and θ values, and the value inside each cell is increased according to the number of co-linear points in Cartesian space. However, for practical purposes, this algorithm is too slow for realtime image processing. Therefore, we must use the Matlab Matlab ‘hough function’ function ’ for our linelinedetection [2]. Once the accumulator is filled, we look for the maximum cell value stored in the accumulator and its related R and θ. The resulting R and θ represent the line we are interested.
The most efficient way to implement these equations is to use a formula node.
Once the line is generated in the Hough Transformation the line values are sent to a
IV. Results and Performance Analysis
In
order
to
test
the
reliability
and
line detection algorithm to determine how to
performance of our algorithm, we carried
properly handle the possibility of crossing
out a series of tests with different scenarios.
the line.
The results were captured and are shown
F. Line Detection Algorithm
below; for each set of conditions there is a
Once we get the values of x, y, x 1 and y1, we
use
the
following
line
picture of what the camera sees followed by
detection
what the program interprets as the proper
algorithm to calculate the line angle. This
avoidance maneuver along with where the
will be used to determine whether the robot
nearest line is.
needs to turn, along with what direction and intensity the turn should be made. If (x > 0) AND (y > 0)
Yes
No
If ( α < 30) AND (α > 0)
If ( α < 30) AND (α > 0) No No Yes Yes
Go Straight
Turn Left
Go Straight
Turn Right
Right = x1 > 160 Left = Not (Right)
Figure 12: Turning Algorithm Right will tell us if the detected line is located on the left side of the camera and Left will tell us if the detected line is located on the right side of the camera. These are decided by what x coordinate pixel the nearest line is on at the bottom row of the image.
Test 1: Simple Right
Test 2: Right w/ Obstacle
Test 3: Parallel
Test 4: Simple Left
Test 5: Left w/ Obstacle
the system would need the guarantee that
According to the test results, we found that
either the left or right line would be in the
our new algorithm can give more accurate
field of view. For our purposes this is not an
and reliable results than our old algorithm.
acceptable loss, but the improvements made
In addition, since we do not use NI Vision
are
Development, and wrote the whole project
improvement over the previous prototype.
with the intrinsic LabVIEW functions, we also solved the problems related to software expiration. One problem that still needs to be dealt with is if no line is present the adaptive
threshold
will
give
the
line
detection of the largest set of intensities. This has to be solved before the system can be declared a fully functional obstacle avoidance utility. V. Further Improvements
Although our algorithm is satisfactory to some extent, there is a lot to be improved upon that would require more time and a budget plan for additional equipment. First of all, we use monocular vision system, to detect the lines. By adding a second camera the system could be reprogrammed to have one camera handle the left and the other dedicated to the right line. This would allow greater control due to the visual field being doubled. For the project to be a feasible substitute for the current system the algorithm will need the ability to distinguish whether or not a line is even present. Without this ability
still
enough
to
show
definite
VI. References 1. Davies, E.R. Machine Vision . 2nd ed. San Diego Academic Press., 1997. 80-269. 2. González, Rafael C.; Woods Richard Eugene; and Eddins Steven L. Digital Image Processing with MATLAB . Pearson Prentice Hall., 2004.380-406. 3. Jahne Bernd. Digital Image Processing . 6th ed. Heidelberg: Springer-Verlag., Springer-Verlag., 2005.331-340. 4. "NI Vision Acquisition Software." National Instrument . 30 Nov 2009. http://sine.ni.com/psp/app/ http://sine.ni.com/psp/app/doc/p/id/psp-394 doc/p/id/psp-394 > 5. Painter, James G. Vision System for Wunderbot IV Autonomous Robot. Elizabethtown College, 9 May 2008. 6. Aung, Than L. & Conlow Michael. Alternative Vision System College, 9 Dec 2009. for Wunderbot V. Elizabethtown College,
