Traffic Quality Management Karl Ernst Ambrosch talks to ITS International about a new KPI-based methodology for assessing the efficacy of electronic toll collection schemes
The debate over which is the ‘best’ solution for applications such as truck tolling is now years old.
A decade or so ago, it was a rather straightforward discussion of DSRC versus GNSS (‘tag’ versus ‘satellite’) technology and the relative merits of thick- and thin-client OnBoard Units (OBUs). The intervening years have served to widen the issue, however, with hybridisation, increasingly sophisticated applications involving more than just corridor charging and smart devices all adding to the mix.
But there is more to a successful truck tolling scheme. Proponents on both sides of the now-legacy DSRC/GNSS debate will argue vehemently over capital and operational expenditures and the overall return on investment of the competing technologies while sometimes failing to appreciate the other factors with potential effects on success and profitability. And as the truck tolling environment has matured and become more sophisticated, we have seen the wider ITS sector move from research and development phase into live deployment. The emphasis is more on real-world procurements and how to justify them in budget-conscious conditions.
The rise of KPIs
The application of key performance indicators (KPIs) has become an increasingly popular method of proving the worth of ITS investments but their success hinges on selecting the most appropriate performance criteria – that will demonstrate ‘success’ in the truest sense. It also relies on accurate performance data – something which is not always easy to come by.
Bratislava, Slovakia-based consultancy Traffic Quality Management (TQM) has just completed a KPI-based study of European truck tolling schemes. Entitled KENTTS – KPIs of European Nationwide Trucking Tolling Schemes, it looks at the relative performances of national schemes in seven European countries.
Karl Ernst Ambrosch, TQM’s managing director, says the KENTTS study grew out of his own background in quality management for the road and rail transport industries. The KPI approach used enables comparisons between quite complex Electronic Toll Collection (ETC) schemes and can be used both to influence how existing schemes evolve and the choices of solutions for schemes yet to be unveiled, he adds.
“In quality management you need to measure parameters related to quality and permanent improvement of quality. Monitoring is needed if optimisation is to be achieved. With something such as an ETC scheme, there are complex, inter-relating issues and, as such, it’s not simply a case of ‘turning the screw’ and expecting improvements to result.
“Since the beginning of ETC there have been numerous attempts to compare the different technological solutions and the trustworthiness of suppliers, as well as the commercial and operational performances of nationwide truck tolling schemes. We can’t, though, compare apples and oranges so a methodology has to be found that can meaningfully compare systems which are different by concept and in terms of technology, purpose, size, framework and regional requirements. KPIs are an effective tool for doing this.”
The various aspects of an ETC scheme are shown in Figure 1.
For truck tolling, other parameters to be defined and considered include the region and network subject to tolling and the tariff scheme. There is also the political decision about whether fees collected are tolls or a tax (which in turn dictates which organisation(s) can collect them and whether, in the case of a toll, value-added tax applies).
The KENTTS study takes a novel approach by also using public socio-economic data provided by the stakeholders’ ETC systems. KPIs are grouped under four main ‘themes’:
Financial Aspects; Set-up and Operation; System Properties; and Socio-Economic. KENTTS focuses on the realisation phases of a scheme – that is, the period from when a ‘go’ decision is taken on a scheme’s implementation. This, Ambrosch explains, takes account of the fact that the need to consult with all relevant stakeholders means it can take several years to reach an initial decision.
“But once a business model has been agreed and a tendering process has taken place, we can usually expect deployment and go-live to take about a year. It’s also important to have an information phase, such that both stakeholders and customers know what to expect from the ETC scheme and what their roles will be. Misunderstanding leads to obstacles and delays, and delays mean cost in the form of additional effort and lost revenue,” he states.
The KPIs themselves
The challenge for complex systems which exhibit non-linear behaviour is to define the most relevant KPIs and the KENTTS study’s KPIs have been defined from the viewpoint of a potential buyer, operator or decision-maker. There are three KPIs relating to Financial Aspects - the first being the percentage of toll evaders, and clearly a low figure is desirable. The second is the ratio of the costs, capital and operating expenditure, to annual income (figures which, Ambrosch notes, some scheme operators work hard not to publicise too freely but which can be had through persistence). The third relates to system price per network length – a more sophisticated metric than mere network length as was used in past comparisons.
In setting up an ETC scheme, several of the most important aspects are time related and a second group of three KPIs, for Set-Up and Operation, deals with this. A fourth KPI relates to the time taken to set up a scheme and the fifth to any delay in the start of operations. The sixth relates to any enlargement of a scheme, whether geographically or in numbers and types of vehicles using it. Enlargement while in operation is challenging but a good performance figure can be arrived at by describing planned enlargement as intended maximum network length divided by initial network length. Higher percentages are preferred.
A well-constructed ETC system should also be simple to use and therefore encourage uptake - failure to achieve this adds both cost and complexity. The KPIs relating to System Properties emphasise a lack of complexity. The seventh KPI, for example, gives preference to tolling schemes that do not need a parallel scheme (such as ETC plus manual toll collection). The eighth looks at passive and active interoperability – the number of countries in which bilateral and multilateral system interoperability is possible or achievable, with higher numbers preferred. The ninth considers the multilingual element, information provision and accessibility for non-domestic users, and in doing so rewards accessibility.
The 10th and final KPI, related to Socio-Economic Data, considers operational efficiency. This, Ambrosch notes, is usually defined as the relationship between income and collection effort but such an approach does not take into account the variation between countries’ relative economic strengths and the relationships between wages and the cost of living. It costs more, for instance, to use tolled roads in Austria than it does in Germany, so a straight comparison between the two is difficult. KENTTS therefore uses a combination of the average toll rate per kilometre and national GDP per month/per capita to give standardised and comparable values per 1,000km of travel with a higher mileage denoting a more affordable toll scheme in socio-economic terms.
To allow comparison, each individual KPI is then normalised, with the poorest-performing scheme providing a zero value and the best-performing rated at 10. The others are then ranged between, and the same process is repeated for each KPI.
First – and lasting – impressions
For the study of European truck tolling schemes it was important, given the relatively small number of suppliers of technology, to adopt a neutral, equidistant stance, says Ambrosch. As such, TQM approached the KENTTS study with no preconceptions of what constitutes a ‘good’ scheme; the most important point was to come away from a technology-centric position and consider performance in a much wider context.
To give but one example, GNSS technology has matured to the point where an argument over whether DSRC is somehow better is irrelevant as the issue is one of application, not whether the technology ‘works’. Once in operation, GNSS technology works well… but GNSS schemes do have a tendency to suffer delays at the deployment stage and this leads to lower scores in the KENTTS study.
“What we have done is come up with an open-source method of evaluating ETC schemes,” says Ambrosch. “It’s available to anyone who wants to use it, although we also offer TQM’s expertise to those who want it. The KENTTS study considers ETC truck tolling schemes from a given perspective but there may be reason to consider schemes’ performance from the viewpoint of other stakeholders for whom different criteria, such as environmental impact, quality of life or commercial issues, may be more important.
“But whichever criteria are considered the most important, the overriding aim should be a smoothly operating scheme which offers service on a 24/7 basis”
