Camera technology a flexible and cost-effective option

Perceptions of machine vision being an expensive solution are being  challenged by developments in both core technologies and ancillaries. Here, Jason Barnes and David Crawford look at the latest developments in the sector

A notable aspect of machine vision is the flexibility it offers in terms of how and how much data is passed around a network. With smart cameras, processing capabilities at the front end mean that only that which is valid need be communicated back to a central processor of any description. That has direct effects on the amount of bandwidth needed and is just one aspect of the technology which has allowed implementers of  machine vision to make purchasing and operational decisions which play off individual cameras’ costs against those of data transfer, and in doing so address assertions that the technology is expensive.

‘Expensive’ is in any case a term wide open to interpretation, says 65 Imagsa’s Xavier Miota. He argues that in times of economic crisis the ITS sector should look to invest more, not less, in smarter technologies: “Cost has to be seen in context. Traffic congestion has a very big effect on a country’s economic performance and investing in technology costs less than investing in people.

“Smart camera technology can be used for many different applications, from the simplest such as data collection and travel time information provision up to the more complicated such as enforcement of driver discipline. Especially in cases such as the latter it can offer a very swift return on investment.”

Flexibility and adaptability are big themes, he continues, with the ability to support multiple applications from a single device being readily embraced in developing countries – something which he feels that developed countries should look at and learn from. “The applications set will only grow and smart camera  technology will continue to make its presence felt in the market.
Not only does it save money in operation but it reduces cost at the installation stage.”

A brief lesson in history

To gain an impression of just where machine vision has progressed to already, it’s necessary to look back, according to 290 Aldis’s Bill Sowell. Some 21 years ago, whilst a product manager at 1763 Econolite, he made the first-ever sale of an Autoscope-based video detection system. That first system, he notes, cost $42,000.

“Now, video detection is an accepted detection standard, and Aldis and its competitors would expect to outfit a four-approach intersection for $15,000 or less. The first-ever market analysis for video detection dates back to 1990 and found that the price point to get to in order to replace loops was around $10,000. We’re there now – or very close.

“Today, like 21 years ago, the installation of a single loop plus amplifier electronics costs about $2,000. But go to a location with extremes of climate and longevity is a problem. That loop will last maybe six months in Riyadh. Southern California expects to get more than four years from a preformed loop set in concrete. That’s $2-3,000 on repair or replacement every few years, plus the costs associated with lane closures and police overtime.

“That gives rise to the issue of lifecycle costing. Too often it’s not considered, with capital and operational budgets being administered by different individuals. One guy is going after the lowest possible installed price while the other is trying to retain as large a budget as possible.”

That situation is improving, however Sowell remains unconvinced that the value proposition of machine vision is yet fully understood.

“Traffic engineers have one thing in common: they want to clone themselves and stand on every street corner so that they can verify in real time what’s going on. They may not be concerned with data collection but affiliated groups might be. The police, for instance, want to quantify incidences of violations so that they can target countermeasures with greater effect.

“We still have a lot of organisational stovepipes, a reluctance to share data and a situation where the capabilities of machine vision are not being effectively utilised. For instance, those same police personnel might deploy surveillance cameras which in truth are redundant because a machine vision system can do the same job perfectly well. Education and awareness would result in the sharing of purchasing and benefits.”

Looking back, in light of over two decades of experience, it  might seem that something has gone awry but Sowell doesn’t think so.

“Nothing’s gone ‘wrong’, as such. Machine vision is enjoying organic growth of more than 10 per cent per year, either as new or retrofit. More and more people are seeing its benefits for applications such as automated incident detection. We’re talking about a traffic-related market worth $300 million a year.”

Steady progress

In many respects Sowell sees machine vision as a well-defined and established market, even if it has become something of a buzz term. The conservatism of traffic engineers of two decades ago has been replaced by the technology-savvy outlook of computer boomers, with the result that sales cycles have shortened and tenders invite solutions which propose loops and/or type-approved video-based systems.

“There’s no debate over whether machine vision is acceptable in terms of accuracy; it is. But we need greater understanding of the full feature set,” he notes.

Over the coming years, the cost of implementing machine vision will continue to decline, he adds.

“Already, it’s pretty much a trade-out but prices will fall as installations increase. Machine vision actually saves money but I’d like to see better appreciation of lifecycle costing and inter-agency cooperation.”

Supporting such opinions is the fact that machine vision is a very fast-developing sector. What follows is a comprehensive
look at factors which are driving the technology’s increasing utility, as well as some comment on where applications are heading.

Standards: an un-ending  evolutionary process

Innovations such as GigE Vision have done much to standardise the high-speed transmission of data over Ethernet networks but there is still much to be excited about where standards are concerned.

USB 3.0, the latest revision of the ubiquitous standard for computer connectivity, increases even further the flexibility which machine vision offers, says 822 Stemmer Imaging’s Steve Hearn: “USB 3.0 offers almost 5Gb/s, more than four times the data capacity of GigE Vision. You don’t get the huge cable lengths possible with GigE Vision but lengths of up to 20m are achievable. With USB 3.0, what you get is scale-ability from ‘dumb’ right up to very high-performance cameras. It allows you to either put a lot more cameras on a bus or use higher-resolution, higher-speed cameras. That’s a very potent mix.

“Machine vision is in many respects all about the camera plus the network, and there’s a lot of use of standard IT and consumer interfaces. GigE Vision, for example, is really only a combination of Gigabit Ethernet with additional machine vision protocols.

There are already USB 3.0 products on the market but we’re going to see the same happen with USB 3.0 as happened with high-capacity Ethernet: the emerging USB Vision standard will use core USB 3.0 technologies but again will add protocols for machine vision. What you’ll get is guaranteed performance and, because it’s being developed by the same people who developed GigE Vision, the standards will complement and not conflict. We can expect to see something formalised by the end of this year.”
In terms of application, that is really quite exciting. For example, in a situation where more pixels or 360^o coverage are needed, 20 to 30 relatively low-resolution cameras could be arranged on a USB 3.0 bus whereas once over the only way to achieve the same result would be to use, say, an expensive 10MP camera.

“That means you can use low-cost cameras and lenses but still achieve the massive amounts of data you might want or need,” Hearn states. “The massive bandwidths also mean that you can run something like a 4MP camera at frame rates in the hundreds per second range, should you need to.”

Dutch company 5854 Adimec’s approach to traffic management is strongly influenced by its experience of the defence sector, where sophisticated electro-optical systems are essential for ensuring both the safety of military personnel and the effectiveness of weaponry. A common requirement is the need for the maximum quality of detailed data that can be grabbed from an often complex scene, as evidenced by moves towards cameras with digital full HD resolution.

These bring their own issues for applications in both sectors. For example, systems have to operate in harsher external environments than many standard machine vision products. And while a data rate of 60fps offers critical advantages for high-speed highway traffic applications, the resulting data stream is greater than either analogue or GigE Vision interfaces can handle.

Adimec has therefore developed and introduced, as a member of the CoaXPress consortium, a new digital interface, CoaXPress. This became a standard in the first quarter of 2011 and is being hosted by the Japan Industrial Imaging Association (JIIA). It can transmit 6.25Gb/s over a single standard coax cable with a cable length of up to 144m. Adimec’s Joost van Kuijk sees the CoaXPress approach bringing overall system cost reductions from the scope for simplified integration and reduced cabling (with power, I/O and commands over a single cable), re-use of existing analogue cabling, and (since one 2MP unit can replace two or more standard-resolution ones) reductions in numbers of cameras.

Consumer versus commercial versus custom…

As in any market, buyers get what they pay for, says 593 Teledyne Dalsa’s Manny Romero. Standards such as GigE Vision have
resulted in standardised, commoditised machine vision solutions which are perhaps 40 per cent cheaper than even three years ago. Nevertheless, the ITS sector’s price reference point for camera  systems is still lower and customers – in this case, those who sell on to the end-user – continue to look to drive costs down even further.

There has been a very aggressive push on the individual component side of systems, he notes. The ITS sector in particular, as distinct from the manufacturing processing industry in which machine vision has its roots, has been shopping for lenses at the commodity end of the market.

“The lens is the most costly component of a machine vision system,” he says. “A typical lens for machine vision applications might cost around US$2,000, whereas consumer-grade Canon or Nikon ef or ef-s lenses cost about a fifth of that. We’re seeing ITS users who want to leverage the cost difference which the high-volume SLR camera lens market offers, or else leverage the full SLR camera itself and adapt it for machine vision applications.”

Although that has a positive – for the buyer – effect on up-front purchase costs, there are pitfalls. Chief among them is that such solutions are consumer- and not industrial-grade. They don’t, therefore, have the robustness of operation across such a wide temperature range and the data flow rates are inadequate for very high-end applications. There is also the high turnover of products in the consumer electronics sector to contend with.

Romero sees such solutions as being adequate – but no more – for portable and mobile applications. They are unable to stand up to the rigours of permanent installation at the roadside. He anticipates that any such solution has a lifespan of 18-24 months, whereas robustness and long-term stability of both the product and its outputs remain focuses for many in the machine vision sector.

“Some customers have been very happy with SLR solutions but they tend to be smaller players looking to compete on a price basis with the bigger players. The solutions have turned out to be cheaper in every sense of the word. Some of the bigger players have also gone down the same path but are now looking to engineer their way out of the situations they’ve found themselves in,” he adds. “I see such solutions having a continuing but decreasing market share over the next couple of years.”

Another way to reduce cost is to go for customised or semi-customised solutions. For certain customers, Teledyne DALSA has taken general-purpose machine vision cameras and then, quite literally, shaved off the un-needed functionalities. This reduces cost and results in a very rigid solution but is a way for system suppliers to be able to provide machine vision performance at less than the ‘full’ price.

“This is in many ways the most usable solution, as you’re not having to front up US$0.5 million for design,” Romero explains. “We use a standard product as a reference and then customise it according to need. We’re seeing more and more solutions like that, not least because a standard product allows customers to trial before they buy.”

A semi-customised product still needs a certain volume to be viable. The exact number depends on the camera and component manufacturer but the numbers can be relatively low – around the 100-150 mark, which makes it a viable course for integrators in the ITS field.

“A general mistake is to try and decouple all components and make each reach a target price. You can’t do that and expect success,” Romero adds. “At the end of the day, customers need to decide what they want a camera to do. If they want the front end to achieve a certain price point, then they have to look at how much processing needs to take place there.
“Take image compression: do you do that in the camera or on the PC? A 12MP image in a good dynamic range needs 12Mb per image just for monochrome. If you want colour on a PC, that’s 24Mb per colour plane transferred. If you need to transfer a lot of images within a short space of time, you might want to compress in the camera instead. Whether something needs to be done in real time or not is a big factor but the semi-customised option represents the best cost-conscious choice.”

Adimec’s Joost van Kuijk also highlights the trend. Like Romero he notes that there is the choice between custom and off-the-shelf solutions. The immediate reaction to the term ‘customisation’ tends to be that it will automatically result in a more expensive product but van Kujik believes that careful analysis of the total cost of ownership can show the opposite to be true. He suggests that existing system redesigns, workarounds and software adjustments to compensate for inadequate original quality and functionality are often inefficient in practice and costly in the end.

“Build-it-yourself may sound good at first, since it gives total control over the product but if the need is for leading-edge technology then working with a camera specialist can save huge amounts of time by eliminating the learning curve.

“Using an optimised camera from the start allows more focus on core technologies and eliminates the need for additional design work. Given lengthy product lifecycles in users’ industries, custom cameras also provide better continuity and obsolescence management.”

CCD and CMOS: the old and the (increasingly competent) new

Cost is also a contributory factor in the battle for supremacy between long-established Charge-Coupled Device (CCD) and newer Complementary Metal Oxide Semiconductor (CMOS)-based cameras. The latter are already showing cost benefits in consumer applications and the effect is now coming through in the industrial machine vision sector.

In the past, CCDs have been considered superior because of their quality, notably in offering a higher dynamic range and resolution. But CCD frame rates are slower, they require more power dissipation, and they cost more to manufacture and integrate.

Recently, CMOS sensors have shown significant improvements, notably higher speed performance, better integration potential, lower power needs, and sensor resolutions and data qualities which are approaching those of CCDs.

CMOS sensors’ development speed is faster, since they are produced in non-dedicated fabrication laboratories, allowing prototypes to be progressed and incorporated into cameras more quickly. CMOS solutions are already very viable alternatives and are likely to outperform their CCD rivals in two or three years – initially in high-speed and later in lower-speed applications, according to Adimec’s Joost van Kuijk. Even in just the last few months we have seen some significant developments in CMOS-based technology, says Stemmer’s Steve Hearn. Like van Kujik, he notes that high dynamic ranges are combined withgreater ease of manufacture: “With CMOS it is, for instance, possible to have all the necessary capabilities including processing on a single chip, whereas with a CCD-based solution a sensor set and then add-on processing is the norm. The effect with CMOS, therefore, is an overall downward pressure on unit price – a trend which I expect to continue.”

Making light of things

Stepping away from the camera slightly, evolutions in lighting are worth noting, says Teledyne DALSA’s Manny Romero: “Developments in LED-based white and infrared light sources are lifting the pressure on the machine vision end of the solution. We’re seeing newer, more powerful light sources becoming available at a decreasing cost. More capable light sources mean that the lighting systems can either be farther away from the camera itself, or else the acquisition process is faster.”

Stemmer’s Steve Hearn notes that with some enforcement applications processing cannot be done on a single image. For example, an image which shows a vehicle will saturate on the license plate. That results in a need for two cameras: a monochrome Automatic Number Plate Recognition (ANPR) camera and then a second (often colour) to provide contextual information. However, multi-pulse lighting systems allow two images of different intensities to be processed of the same vehicle; with machine vision, the ability to decide frame by frame and at electronic speeds which images are required means it is possible to use just one camera.

“You can use IR flash to make an ANPR capture and then just a fraction of a second later trigger another flash at a different intensity to get the overview image,” he says. “The key is having a high enough frame rate but you don’t just remove the need for one of the cameras; you also potentially remove the need for an external trigger as the trigger itself can be vision-based.

“That has associated installation and maintenance savings. There are many other applications where the inherent intelligence of
machine vision can have the same effect.”

LED technology manufacturer 6965 Gardasoft Vision has made something of a specialism of getting the lighting sources for ITS
machine vision applications right. A driver of that strategy has been a realisation that simply picking machine vision out of applications in manufacturing processing and dropping it into ITS will not work, says the company’s Peter Bhagat.

“That we’ve been very successful in machine vision applications elsewhere doesn’t matter. We’ve still had to take our core technologies and adapt them more closely to the applications and requirements specific to ITS,” he states.

There are a number of areas where LED lighting for vision applications has and will continue to influence innovation in ITS. Multi-pulse LED lighting systems which enable cameras to take several images with different pulse lengths and software which automatically chooses the best one(s) for the application are a departure from the convention, where only two images are taken and then compared. Generally, processing requirements are reduced if the lighting conditions are better and Gardasoft provides solutions which can provide feedback from the camera to the lighting system in real time via Ethernet or RS-232 and take account of time of day and prevailing conditions.

“Intelligent, dynamically controlled lighting reduces complexity and cost by, for example, reducing the need for auto-iris functions,” Bhagat explains.

Echoing Hearn’s sentiments, he adds that LEDs score over xenon-based technology because they allow faster pulsing,
typically 30Hz versus 2Hz. That allows detection and triggering to be accomplished via the camera, removing the need for an external sensor, such as a loop or laser. Many systems now trigger the camera continuously and use the images to detect whether a vehicle or other object of interest is present.

“That’s a further cost reducer because there are no or fewer civils involved in installation and the on-going maintenance burden is much less. LEDs themselves have no consumable parts, which further
reduces costs.”

Applications: giving enforcement agencies a sporting chance

Stéphane Clauss of 4551 Sony Europe’s 5853 Image Sensing Solutions (ISS) division sees abundant scope for highly compact cameras, as exemplified by the recent addition of the FCB-SE600 colour module to its range of FCB block range cameras. The new arrival incorporates a 1/2.8-type Exmor CMOS sensor designed to deliver full HD resolution (1080i/60fps) and a high level of sensitivity in environments with light levels as low as 0.1lux. Developed in collaboration with the Sony Exmor team, it measures 50x47.6x54.2mm, suiting it to space-restricted environments at the roadside (for example, for access control) and in-vehicle (for use in public transport vehicles for passenger security monitoring and counting, and for schedule planning).

Clauss foresees growing markets within the transport security market. Major users to date include three French national security organisations – the gendarmerie, the police, and the customs authority – who have equipped 1,000 vehicles with an integrated Automatic Number Plate Recognition (ANPR) system designed by systems integrator Survision to fit inside the limited space of their vehicles’ rooftop light bars. The system incorporates seven Sony FCB-EX20DP cameras together with illumination and processing equipment.

Major sporting events are proving to be a rich market. For the London Olympics, ANPR system specialist NDI-RS is working with London boroughs on the deployment of camera networks using its NDI-RS-C320 system to monitor critical routes in and out of the Games’ home zone in Stratford.

At the heart of NDI-RS systems are two high-resolution Sony FCB block cameras. These output images in both infrared (for ANPR capture) and colour (for evidential overview imaging) to the London Metropolitan Police data centre, which runs NDI-RS’s TALON technology.

Brazil is getting ready to stage the Football World Cup in 2014, followed by the next Summer Olympics in 2016. A third of the
vehicles in the country have been brought in illegally, causing a major security problem, and the authorities are using machine vision technology from Curitiba-based FiscalTech to monitor roads. Its handheld RVG-Speed Control device embeds a Sony high-resolution camera with laser speed detection, wireless connectivity and an Optical Character Recognition (OCR) system.
Claimed to be the smallest of its kind, the scanner can monitor up to four traffic lanes simultaneously for traffic volumes and automatically identify vehicle type and colour.
But in many respects, we’re only just beginning to see what this technology can do for the ITS sector, says Gardasoft Vision’s Peter Bhagat: “Specialist applications are now becoming a reality. For example, by having a series of high-quality images taken from different angles, it becomes possible to profile a vehicle, not just identify it by its number plates. That has big implications for security applications. Such things are already being worked on, and by the types of major players who have the ability to get them to market within a relatively short time from now.”

Already, some ANPR cameras can provide more information than previously thought. Identifying a vehicle involves more than just discerning its license plate. Other identifying features such as car make and model provide much-needed information to law enforcement and traffic management personnel. Today ANPR cameras can be self-contained edge-of-network computing units with the ability to provide much sharper images at higher resolution, which provides much more visual
information, and the algorithmic ability to analyse that information automatically, providing data that was once the sole purview of video analytics.

To address customers’ demands for more information from a single system at lower cost and high efficiencies, Israeli company 4015 HTS offers a vision-based Vehicle Recognition System (VRS) which reads and identifies plate numbers and letters, vehicle make and model, and vehicle speed. The information is recorded in real time for further processing according to customers’ needs. These systems have recently been deployed in Mexico where law enforcement customers have stated outright that license plate information, as important as it is, is simply no longer enough and that they will not spend money on a standard system that does not scale for other applications.

The VRS’s recognition capabilities enhance and improve vehicle verification and classification and help check correlation between the vehicle type, license plate number, and data stored on police and homeland security databases. The ability to generate immediate alerts improves reaction times to crimes committed. This includes amber alerts in cases of kidnapping.

Less dramatic applications are also benefiting from machine vision technology.

The population of Dubai, in the United Arab Emirates, has more than doubled over 10 years to nearly two million in 2011. The number of vehicles on the road now totals just over one million. That has created a huge demand for parking capacity.

To cope with current levels and expected growth, the 6700 Dubai's Roads and Transport Authority (RTA) announced in 2010 a plan to build 50 new parking garages, largely in the central Deira and Burj Dubai districts, each with a centralised automated payment system. The brains of this, the HNAPR automated machine vision-based ANPR solution developed by locally based Niaar ITS LLC, uses a 2255 Prosilica GC1290 camera from Allied Vision Technologies, a 1MP digital unit with a GigE Vision interface and a Sony ICX445 ExView HAD sensor.

The system is designed to perform accurately at distances of up to 15m in adverse conditions. A central requirement from the RTA was for simplified and speedy management of vehicle entry and exit transactions to optimise gate management and limit waiting time, so reducing on-street traffic congestion. Storage of captured licence plate image data, with each plate character defined in the software and identified using OCR, together with referencing time and date information, provides statistics for traffic level analysis and can restrict parking usage to certain vehicles based on date or time of day.

The camera is connected to an industrial PC running Windows across an Ethernet network via Cat6 Ethernet cable or fibre optics, with a server located at each gate entrance. The company claims that the system can be installed in less than two hours and is low maintenance, needing six-monthly re-calibration to maintain an accuracy exceeding 95 per cent.

The ‘region of interest’ capability of machine vision emerges in a system which has been developed by US company Transport Data Systems (TDS) to capture and decode the official identification numbers that the 324 US Department of Transportation (USDOT) requires commercial interstate freight operators to print on the sides of their trucks.

State DOT workers rely on computer terminals to check carriers’ safety records using the USDOT number but this typically occurs only during random checks or when a truck has violated a weight restriction, for instance. The new system is designed to work 24 hours a day, in variable outdoor lighting conditions, capturing vehicles moving at very high speeds.

Once the image is captured, the reader extracts the USDOT number and cross-references this against federal agency databases, with the results accessible via a browser and stored for future tracking. To achieve the necessary quality of image capture, the camera had to offer: high sensitivity, to ensure the short exposure time necessary to eliminate motion blur cause by vehicles speeds of over 96km/h; a large field of view in order to contain the whole USDOT number in a single frame; high resolution, to ensure reliable OCR readings; and a high dynamic range to distinguish key image features in all weather conditions.

TDS chose 541 Point Grey’s Grasshopper 5MP camera model GRAS-50S5M because of its ability to reduce the vertical resolution of the field of view to allow the capture of relevant data only – in this case, the side of the truck.

Each installation varies, depending on the lane geometry of the inspection station, but in a recent DOT installation in Montana trucks pass through a series of sensors along the access road to an inspection station. Signals from these are relayed to a USDOT reader, which in turn triggers the Grasshopper camera; this uses optically isolated hardware trigger output to strobe TDS’s custom-made infrared LED illuminators to provide optimal illumination for accurate OCR reading.