Please note that JavaScript and style sheet are used in this website,
Due to unadaptability of the style sheet with the browser used in your computer, pages may not look as original.
Even in such a case, however, the contents can be used safely.
Volume 63 (Dec 27, 2006)
The IMAPCAR® parallel processor for image recognition and its contribution to the realization of pre-crash safety solutions for automobiles
Sensing danger even before the driver can
Nearly every driver has at one time or another slammed on his or her brakes to avoid an obstacle on the road and then breathed a sigh of relief after having successfully evaded an accident. It is said that the stopping distance when braking suddenly is 22 meters (m) at a speed of 40 kilometers per hour (km/h), and 44m at a speed of 60 km/h. This means that regardless of the level of brake performance, the sensing of danger even before the driver can respond is integral to the prevention of accidents.
Pre-crash safety functions have therefore recently become a popular topic of discussion. Using information obtained from built-in cameras and milliwave radar systems, the positioning of obstacles, including pedestrians, as well as the distance between the vehicle and the obstacle in question can be detected. In the event that the probability of a collision is determined to be high, either a warning is issued to the driver, or the brakes are automatically applied by the control system, thereby preventing an accident.
To recognize pedestrians and other obstacles, a high-performance image recognition LSI chip capable of real-time processing of images captured by cameras is indispensable. In the future, automobiles will be capable of recognizing an even wider variety of obstacles located in their environs. NEC Electronics has therefore developed IMAPCAR, a parallel processor for image recognition that is ideal for image processing for automobile use (Figure 1, Photo).
The long road to the realization of 100 GOPS performance
The performance of IMAPCAR is expressed using a unit referred to as 100 giga-operations per second (GOPS). This unit is used to describe processing executed at a rate of 100 billion operations per second. According to Fumiyuki Hidano, who was involved in the development of IMAPCAR, "Although it of course depends on the application being implemented, IMAPCAR is easily capable of recognition processing for all frames of ordinary video input images (30 frames/second) when it comes to the recognition of lane markers and forward vehicles". Figure 2 provides a concrete example of this capability. In the event that a vehicle is traveling at a speed of 60 km/h (assuming that the traveling distance per second is approximately 17m and IMAPCAR has a processing capability of 30 frames per second), IMAPCAR will perform moving image recognition of scenery viewable from within the vehicle every 0.5m.
However, the realization of 100 GOPS performance did not come easily. When parallel processing architecture was invented using an integrated memory array processor (IMAP) back in 1990, it was not actually meant for automobile use, but rather purely for research and development as an "image recognition technology" (Figure 3). In 1995, IMAP-VISION, which boasted a 256 parallel, 20 GOPS processing capability, was developed. It was from this that the foundation for image recognition began to take shape.
Then, in what Hidano describes as being "a fateful encounter with the automobile," NEC in its entirety embarked on the ITS Project in 1996, and in parallel with this research, pushed forward with related research in the automotive field, including lane-marker recognition, leading vehicle recognition, recognition of adverse weather conditions and more. "At first, we were faced with the issue of how to adapt this technology for automotive use. During the research and development phase, despite the fact that 256 parallel had already been achieved, it was necessary to take into consideration chip size, power consumption, production technology and quality as we worked toward integration onto a single chip."
IMAP-CE, which was unveiled at the ISSCC held in 2003 in the United States, uses an SIMD system for which 128-parallel operating units follow identical commands and a VLIW system capable of simultaneous execution of four commands in one cycle, and comes equipped with 2 Kbyte SRAM for each operating unit to enhance execution performance. These components made it possible to realize a performance level of 51.2 GOPS, which was at the time the highest in the industry, and a one-chip prototype was created. Nevertheless, to actually be adopted as a product, a performance level of 100 GOPS—approximately twice that of the prototype—was necessary.
In refining the architecture established for IMAP-CE, many adaptations, including making immensely difficult changes to the operating processing logic that serves as the core of IMAP, were made to meet the demand for increased performance. Among these adaptations, one worthy of special mention was the extension of some of the 8-bit operations to 16-bit operations in order to ensure performance. In this way, the implementation of various measures enabled the realization of a performance level of 100 GOPS (Figure 4).
The destiny of automotive devices
Due to high automotive safety requirements, strict operating conditions for automotive devices are required to ensure safety, and power consumption is one of those conditions. Heat loss countermeasures become necessary when power consumption reaches 2W or higher, which affects not only cost but also the site of installation. This in turn becomes a factor when making design changes to automotive systems, thus placing an enormous burden on automobile manufacturers. Through the adoption of 130-nanometer (nm) process technology and achievement of a low operating frequency of 100 MHz through parallel processing made possible by high-speed processing ability, a low power consumption level of 1.7W was realized. In addition, the temperature environment of an automobile is far more severe than that of consumer electronics systems, and even for electronic devices, an operating temperature range of -40 to +85 degrees Celsius (+125 degrees for the area surrounding the engine) is desired. Expansion of the operating temperature range was achieved by the aforementioned lowering of power consumption and enhancement of radiation performance through adoption of a high-performance package.
Once the hurdles of high performance, low power consumption, ambient operating temperature and device performance had been cleared, the issues of building test coverage into the mass production process in an aim to attain a defect detection rate of 100% and maintaining quality during the mass production stage remained. Feeling the weight of the responsibility of producing a product related to automotive "safety," the development and production teams joined together in hopes of achieving a market defect rate of zero (zero defects) and implemented the most effective defect reduction measures possible.
And with this, IMAPCAR for automotive use was born. However, it's not simply the fact that IMAPCAR boasts the industry's highest operating performance as an embedded processor that makes it impressive. Since all of the image recognition functions are processed by C language software expanded to include data types, operators and syntax that are optimal for parallel processing, evaluation can be carried out up until right before mass production of a system begins, and modifications, version updates and so forth can be easily made. Moreover, the IMAPCAR development environment includes a software library organized into functional groups that enable faster time to market. These functional groups range from a source-level symbolic debugger equipped with a GUI and video controller to a filter and geometric converter, as well as image input and output ports. IMAPCAR also provides an evaluation board for fast launching of system development environments. These development environments in turn contribute not only to maximization of IMAPCAR capabilities but also to enhancement of development efficiency.
Hidano concluded by saying, "Image recognition algorithms and parallel processing technology as well as software development and evaluation are integral to the development of semiconductors for image recognition use. And with an appropriate support system, image recognition technology that is even more advanced can be put to practical use."
There are no limits when it comes to enhancement of safety. NEC Electronics will continue its efforts to further improve the performance of the IMAPCAR image recognition parallel processor and contribute to the realization of collision-free automobiles and a safe motorized society.
• IMAPCAR is a registered trademark of NEC Electronics Corporation in Japan.
425KB)