Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
A Study of a
173
Image Recognition on Kansei Engineering b
b
T. Jindo, M. Nagamachi, and Y. Matsubara a
Vehicle Research Laboratory, Nissan Research Center, Nissan Motor Co., Ltd., 1, Natsushima-cho, Yokosuka, Japan b Hiroshima University, 4-1, Kagamiyama 1-chome, Higashi Hiroshima, Japan Kansei engineering is a discipline that concerns the creation of products that convey a certain desired image to customers, based on research into the relationships between the perceived impression of a product and its physical properties such as color or shape. This paper presents the results of a study concerning the classification of the features of products which tend to incorporate sp~ialized or subjective elements. An attempt was made in this study to use image recognition technology to judge and classify the physical properties, i.e., design elements, of the automotive steering wheel automatically in order to eliminate any subjective aspects. 1.
INTRODUCTION
The overall framework of this research was defined as the first step toward the development of a system for automatically classifying the design features of automotive steering wheels. As a result, the significance of automatic feature classification and the specific details of the system for executing this task were specified as explained below. The aim set for the automatic feature classification system was not to describe the physical features of an input sample. Rather, it was desired that the system should be able to classify the features automatically into respective categories on the basis of predetermined design elements. Table.1 shows the design elements and categories that were defined for automotive steering wheels in this study. In view of the fact that this study was being done at the research level, it was decided to use a scanner to input two-dimensional images such as illustrations and photographs into the recognition system. On the basis of these two-dimensional images, the system would then automatically classify the steering wheel design features.
2.
DELOPMENT OF DESIGN FEATURE RECOGNITION SYSTEM USING
S T E E R I N G W H E E L ILLUSTRATIONS Template matching is frequently used as a technique for recognizing design features from illustrations of products. This method works if the design in question is among several design patterns known in advance and does not include any displacement or changes. With this method, an input image is compared with templates, which have been prepared in advance and stored in memory, from every possible position. An index of similarity between the input image and the templates is found in every position. At the point where the index reaches its
174 maximum value, a judgment is made that there is a template which matches the input image at that position. This method can also judge a design pattern for which the index of similarity shows a maximum value among several templates. However, one disadvantage of template matching is its extremely high computational costs. Another drawback is that pattern recognition is impossible if the input image includes displacement or modifications.
Table.l.Design elements/categories and related definitions Definition A steering wheel with two spokes A steering wheel with three spokes A steering wheel with four spokes Space between spokes is small Spoke angle Small Space between spokes is large Large Open space emphasis Open space emphasis A type that accents the shapes of the ooen spaces A type that accents the spoke shapes Spoke emphasis or spoke emphasis Pad surface and spokes are continuous Present Spokes and pad Pact surtace and spokes are cttscontmuous ADsent continuity No switches on the wheel Presence of control None Switches are located on the pad Embedded in pad switches Switches are located on the sides of the wheel On sides Horn button is integrated with the pad Horn position Pad type Embedded in spokes Horn button is located on the spokes A type with finger rests Present Presence of fmger A type without finger rests Absent rests Size of side open space is large Large Size of side Size of side open space is small Small open space There is no sicie open space None Pad is large in height Large Pad height Pad is medium in height Medium ?act small in nel~lat bmml Pad is large in width Large Pad width Pad is medium in width Medium Small Pad is small in width None Pad has no pattern Pad surface pattern Circle Pad has a round line(s) Oval Pad has an oval line(s) Horizonta_! line Pad has a horizontal line(s) 2 kinds Pad has 2 kinds of line r'aa area is Large iq,T0IB,!lf~ ~.!~'.4mN Pad area is meoium vM~/lfft~ Small Pad area is small Upper edge has a small radius of curvature Sharp chevron Shape of pad upper ea~e has a large raoms ot curvature tientte chevron upper edge Upper ed~e has an irregular shape Irregular chevron
Element No. of spokes
Category 2
The patterns to be recognized in this work were illustrations of car steering wheels. Since it was assumed that freehand illustrations would be included, it would be impossible to avoid displacement or changes. An effort was made to devise an effective way of overcoming this problem. The method adopted here first resolves the individual patterns that are interconnected within an image into constituent lines. Then, these lines are interpreted on the basis of intrinsic
175 knowledge of each line assumed to be present in a steering wheel image, and that facilitates recognition of the design features. Figure 1 is a flow chart of the design feature recognition process beginning with the input of a steering wheel illustration. The input illustration is first processed to obtain fine lines, and line searches are then performed in order to resolve the image into its constituent lines. This makes it possible to obtain feature information on each line. Using the information thus obtained, a judgment is made as to whether the input image has been rotated. If it is judged to have been rotated, the original image is displayed again and modified to reflect the rotation. Following this correction, fine-line processing, line searches and acquisition of feature information are performed again to facilitate recognition of the design features. The results are then output for display on a computer screen.
(Input steering wheel illustration )
IPe o m fme-lme- o ossing I Perform line search to obtain feature information Procedure when original image has been rotated ~ , , -
on,~.g.y~.~lnfifiOgeO?f~ ~
_
~
I No
~
Display original image again I [ Per f orm correction for rotation of
Io gin age
[Performfree line processing 1 [ Recognition of steering ] wheel design features ~
Perform line L h to obtain ] feature information
IOutput recognition results I
Fig.l.Flow chart of design feature recognition procedure 2.1
Algorithm for design feature recognition An explanation is given here of the recognition algorithm for a case where the original image has not been rotated. The line-search operation resolves the illustration into several lines, and feature information is obtained on each line. At this point, it is necessary to interpret precisely what each line is, in order to support recognition of the design features of the steering wheel. The lines presumed to make up a steering wheel image can be divided into three major types: background portions, the pad surface pattern or horn portion and the boundary lines between the pad and the spokes. Specific examples of these lines are shown in Fig. 2. Recognition of these three types is performed in a series of operations at different levels using intrinsic knowledge regarding the area of each type.
176
t~erin~ wheel image
Level 1
0
0
Fig.2.Resolution of steering wheel image into constituent lines Based on the results thus obtained, the process of recognizing design features proceeds according to the design elements and categories shown in Table 1. An explanation of the procedure is given below. 1. Calculations are made of the center point of all the lines indicated at level 2 and of the distance from the center point of the line denoted as (1) in Fig. 3. 2. Using intrinsic knowledge about distance, a determination is made as to whether the object line is a candidate line for the horn portion. 3. Using intrinsic knowledge about distance and template matching, a determination is made as to whether the object line is part of the pad surface pattem. 4. The remaining lines that are not part of the horn portion or the pad surface pattern are changed to those indicated at level 3. 5. All the lines at level 3 are cleared from the screen. 6. Using the background portions at level 1, the number of spokes is counted. Depending on the number of spokes present, the subsequent procedure of the recognition algorithm changes slightly. The following explanation is for a steering wheel sample having four spokes. 7. The height of the pad is determined from the center points of lines (2) and (4) in Fig. 3. 8. The width of the pad is determined from the center points of lines (3) and (5). 9. The area of the pad is determined from the area of lines (1), (2), (3), (4) and (5).
177 10. The size of the spoke angle is determined from the difference between the point where the vertical axis from the center of (1) crosses (2) and the lowest point of the vertical axis of (2). 11. The shape of the pad's upper edge is determined from the size of the spoke angle and a shape search performed on line (2). 12. The distinction between open space emphasis or spoke emphasis is based on a rule that all four-spoke steering wheels are of the type that accents the shapes of the open spaces.
(1)
(5) Fig.3.Numbers of steering wheel lines
2.2
Recognition results for steering wheel illustrations The above-mentioned algorithm was incorporated into a design feature classification system. Illustrations of steering wheel samples were input into the system and a recognition test of the design features was conducted. Typical results obtained are shown in Fig. 4. As indicated in this figure, the system is capable of displaying an image of an input illustration and the categories recognized for each design element.
No. of spokes ******************************* 4 Pad height ****************************** Large Pad width **************************** Medium Pad area ******************************** Large Spoke angle ***************************** Large Shape of pad upper edge * Chevron with irregular shape Size of side open space ********************Small Open space emphasis or spoke emphasis ************ • *************************Open space emphasis Pad surface pattern ***************************** • *************Oval pattern and two horizontal lines Horn position ***************Embedded in spokes
Fig.4.Typical recognition results for steering wheel illustration The system could not recognize such design elements as the presence of finger rests, continuity between the spokes and the pad and the presence of control switches, as it was difficult to confirm definite information on these elements. With the exception of these three elements, it recognized 90% of the design features of the 30 steering wheel samples. The samples for which feature recognition was not possible or recognition errors occurred tended to have a complex control switch layout or intricate boundary lines.
3.
CONCLUSION AND FUTURE PROSPECTS
In this research, image recognition technology was applied to the automotive steering wheel with the aim of achieving objectivity in the classification of the physical features of
178 products in the discipline of kansei engineering. An automatic feature classification system was developed that showed a high rate of accurate recognition of the limited samples (i.e., illustrations) used in this study, with the exception of some physical features. Recognition test results confirmed that this system has the potential for use in the automatic recognition of physical features in the kansei engineering field. There are still many issues that must be resolved, however. One issue concerning the present recognition system for steering wheel design features is to achieve a higher rate of recognition and the capability of recognizing the design elements that the system failed to recognize in this study. This will require further refinement of the recognition algorithm and the application of more advanced image recognition technology. Another issue is to achieve recognition of design elements based on photographs and other visual images in addition to illustrations. In order to recognize the design features of steering wheels from photographs of car interiors, it is necessary to isolate the steering wheel from the instrument panel and dashboard that form the background. This is especially difficult to accomplish in the case of car interior samples photographed under nonuniform conditions due to shadows created by light coming in through the windshield. The authors intend to pursue the possibility of accomplishing the second issue using samples photographed under certain given conditions, instead of a large number of unspecified samples. Once the second issue has been accomplished, it should be technically possible to discriminate the shape and other physical features not only of the steering wheel but also of the instrument panel and dashboard from photographs of car interiors. The authors intend to pursue the development of technology for automatically recognizing the design features of the entire car interior, and incorporate the data obtained into a kansei engineering system for estimating the overall impression conveyed by a car interior design. REFERENCE
1. T.Jindo and K.Hirasago, A Study of Kansei Engineering of Passenger Cars, Proc' of JAPAN-U.S.A. Symposium on Flexible automation(1994). 2. T.Jindo and K.Hirasago, The Development of Car Interior Image System Incorporating Knowledge Engineering and Computer Graphics, Proc. of IEA, 625/627 (1991). 3. M.Nagamachi, Development of Interior Consultation System using Knowledge Engineering Method, The Japanese Journal of Ergonomics Vol.22, No.1,1/7(1986). 4. G.Strang,Wavelets and Dilation Equations, SIAM Review, Vol.31,No.4, 614/627(1989). 5. T.Yanagishima and M.Nagamachi, Kansei Engineering Approach to Automobile Interior, The Japanese Jornal of Ergonomics Vol.24, No.6,38/40(1988). 6. T.Nishimura and T.huzimoto, Fast Contour Line Extraction Algorithm Selectively Using Threshold Values Observing Line Connectedness, The Transactions of the Institute of Electronics, Information and Communication Engineers, Vol.J76-D-II, No.6, 1186/1193(1993). 7. T.Watanabe and T.Shibata, Detection of Broken Ellipses by the Hough Transforms and Multiresolutional Images, The Transactions of the Institute of Electronics, Information and Communication Engineers, Vol.J73-D-II, No.2, 159/166(1990). 8. Y.Hotta, M.Miyahara and T.Otake, Region Segmentation Image Coding Using the Local Feature of Color Information, The Transactions of the Institute of Electronics, Information and Communication Engineers, VoI.J76-D-II, No.5, 1023/1037(1993).
