Sony’s Arnaud Destruels explains the benefits of CMOS sensors for ITS applications.
In the transport sector roadside, trackside and platform cameras were devices for viewing and assessing a situation while individual sensors did all the clever stuff like traffic counting, speed calculation, queue lengths, signal status and so on. Well, not any more. The advent of vision technology allows cameras to be an above-ground alternative to many individual sensors and opens up new opportunities for smart detection systems that can track Hazchem trucks, pick out blacklisted vehicles on a busy motorway and much more.
A key part to enabling this progress is the ever-increasing performance of camera systems and it is worth looking at how new imaging sensor technologies, coupled with a more holistic approach to system design, are enabling far greater accuracy rates.
Beyond resolution
Ideally, a machine vision system would have a consistent exposure on every part of the object being captured. However, this is not an ideal world and even if you control conditions well, this is difficult to achieve. Some surfaces are reflective and can cause glare while others may have less contrast, making shapes less defined and creating the potential for errors.
This is a problem for all machine vision systems, even those production lines that are indoors and have carefully controlled lighting and filters. If we look at outdoor systems, and in particular ITS, then it quickly becomes apparent that you have next to no control.
The sun’s movements, the quick changes in light conditions that can result from cloud cover or the shadow cast by overtaking vehicles can obscure a licence plate (as well as countless other key elements being read). Additional lighting, the traditional way to cope with this, would reflect from windscreens and licence plates as well as creating further distractions that would reduce road safety.
Added to all those complications, the items you want to capture are moving, sometimes at very high speeds and has led to the demand for high resolution, high shutter speeds cameras for ITS applications.
Resolution does play a key role in capturing the detail required but to capture images effectively requires a truly multifaceted solution that uses a wide-ranging knowledge of how image-processing functions, as well as how to minimise the rate-limiting steps in the system.
New technologies
The move from CCD (charge-coupled device) to CMOS (complementary metal oxide semiconductor) image sensors provides a great opportunity to counter many of these issues and improve the overall image quality.
This is not simply based on the number of pixels available, but other features such as the massive internal parallelism and the associated capture rates that comes with this new sensor technology.
With regard to the parallelism, a CMOS chip with an analogue to digital (A/D) converter at the end of each row of pixels makes it possible to significantly increase the number frames that can be captured each second. This is done without registers and so collects digitised information from an entire column of converters simultaneously so, for example,
This speed means it is possible to create a single composite image from multiple frames which allows a system to, for instance, remove imaging errors caused by heat haze or similar effects. In addition, the high frame rate means the exposure time of each frame in the composite can be subtly adjusted to create a greater dynamic range than would be possible from an individual still image.
This means the composite can use the best exposure level for each part of the image to give the ideal contrast and brightness, without the other elements in the image being under or overexposed. This holds true for images captured indoors or outdoors and allows for greater image consistency on, for instance, tolling and enforcement systems needing to capture number plate details in highly changeable lighting conditions, as a case in point.
Rolling-shutter effect
This is not to say that CMOS is not without its draw backs - in particular, its rolling-shutter design which can compromise ITS-style applications because each pixel is captured at a slightly different time to that of its neighbour. By understanding how the system works it is possible to engineer those faults out of the equation.
Consumer cameras are relatively simple in design because they are not designed to capture items moving at high speed and, if they do, the artefacts created by the rolling-shutter design will not overly-adversely affect how the resulting footage is used.
In machine vision, however, the rolling-shutter design can create time-distortions in the image which can create false positives/negatives. The answer is to apply a global-shutter approach to expose all lines of the image simultaneously and enable each row within a frame to be captured at the exact same time.
Obviously, these performance advantages that come from high capture rates, as well as the global shutter, have to be matched up with other system design choices. Mounting tolerances on both the sensor and lens need to be extremely fine to ensure consistent captures are made and spatial accuracy is kept as high as physically possible.
In short, the success of a machine vision system for use in ITS - and the benefits it brings - is rarely about pixel count alone. Instead, many factors need to be considered to minimise each rate limiting step and get a better overall image that can be used to automate systems efficiently and cost-effectively.
ABOUT THE AUTHOR: Arnaud Destruels is the product marketing manager for vision cameras at Sony Image Sensing Solution Europe.
