U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
At a time when the United States is working to define a Federal-State-regional-local framework for transportation performance management, the international examples examined in this scan hold many lessons. The performance management systems of the studied agencies demonstrated clear linkage between government expenditures and transportation agency results. Long-term government goals were incorporated into transportation agency actions–and the results of those actions could be clearly documented to show what the public received for its transportation investment.
The officials visited offered the scan team invaluable advice from their past decade—in some cases more—of performance management. The systems they developed applied to diverse settings, from the complex and densely populated Great Britain to the rural and isolated New Zealand islands. Despite the diversity of their applications, the performance management systems had five universal components, or steps. From Sweden to New Zealand, the officials interviewed recommended similar processes:
The federal-state relationships found abroad were more akin to coach-player relationships than to umpire-player relationships. It was common to find different levels of government jointly setting a target, then collaborating on ways to achieve it. It was not common to find one level of government setting a target, then penalizing another for missing it.
The scan team found that the true value of performance management was in achieving steady long-term progress. Many officials stressed that another important benefit of their performance management systems was the transparency they created. The transparency improved understanding about transportation issues and led to greater degrees of trust. Striving for long-term accomplishment created collaboration among levels of government, not contention.
Despite the greater linkage of national goals to agency activities found in the visited countries, the central governments set few explicit and quantitative national transportation targets for the transportation agencies. The two exceptions found were in the areas of safety and climate change in the two European countries. The central government articulated broad policy goals, and the transportation agency translated those goals into specific performance measures or targets in collaboration with the federal or state government. This collaborative target-setting practice appeared to exist between both national and state governments and state and regional/ local governments.
The scan team seldom found that one level of government mandated the performance of another. Rather, service-level agreements or other negotiated documents between the central government and the transportation agency were used to define performance measures and targets for which the transportation agency was held accountable. The service-level agreements communicated priorities and clarified outcomes while allowing each state or region to negotiate measures and priorities important to its unique circumstances. These negotiations were supported by extensive data collection that showed trends in systemwide performance. Negotiations between the agencies and their central governments were fluid and continuous. Flexibility was particularly evident with major cities, where unique transportation needs and solutions were recognized.
The combination of national goals cascading into state or regional performance measures appeared to create a strong focus on outcomes instead of process. Not evident abroad were the highly detailed and procedural, fiscally constrained, long-range plans and short-term Transportation Improvement Programs (TIPs) found in the United States. Instead, the negotiated service agreements served to clarify desired outcomes over the next 1- to 5-year period.
The agencies the scan team visited clearly documented system and organizational performance, often in detailed trends over a number of years. The richness of reporting was usually quite sophisticated. The agencies demonstrated improved customer satisfaction, higher reliability in transit and highway travel times, reduced environmental impacts, and greater efficiencies. Their performance management systems naturally dovetailed with asset management systems. The agencies demonstrated a keen knowledge of system conditions and trends and a finely calibrated understanding of system investment needs, often by asset type and region. Clearly, the agencies benefited from managing their performance to maximize resources, optimize assets, and earn credibility from legislators and budgeting agencies.
Despite those benefits to both legislatures and agencies, budget appropriations were for the most part not driven by the resources required to achieve the performance targets. In short, performance management appeared to be less of an influence on budgeting than other factors. Performance management demonstrated how funds were spent and to what end, but the systems did not appear to include a feedback loop that triggered legislative appropriation decisions. The team saw little evidence that legislatures or executive branch financial agencies establish asset investment levels based on data from performance management systems. One British official noted there was no easy linkage between transportation program goals and the budget set by legislators or Treasury officials.
Performance results, however, were considered important in budget discussions. The performance results demonstrated how effectively the agency spent its budget, but the performance targets did not strongly influence the budget level. Discussions indicated that this was because of overall funding constraints in competing public sectors, such as education and health care. It was not because of an ineffective performance management program or agency performance. Most agencies did not have a dedicated road user fund; agencies competed for funding with all other government programs. One agency stated that while its performance management did not garner a budget increase, elected officials viewed it as so effective that it was able to sustain its budget when others were cut.
In three of the six cases, agencies reported discouragement that they could not convince legislators to invest more in system preservation despite their sophisticated documentation. Further discussion noted that identifying large maintenance funding gaps was a longstanding concern. Similar to the United States, the countries had difficulty expressing the impact of changes in pavement and bridge condition at the political level.
Queensland developed a sophisticated means of demonstrating long-term financial risk by calculating the unfunded liabilities caused by underinvestment in maintenance. It uses asset inventories, management systems, and overall asset management approaches to convert asset maintenance needs into a balance-sheet calculation. The process was similar to the Government Accounting Standards Board 34 (GASB 34) process required in the United States. Sweden, the United Kingdom, and New Zealand used variations of this approach to summarize all asset management financial needs in one comprehensive calculation that documented the future infrastructure liability the government faces.
Central government decisions on agency operating budgets tend to remain incremental. System preservation increases were modest and based on increases from past budgets. Budget increases depended on whether the government had any residual revenue left once other priorities were met.
The host countries provided examples of recent budget increases to expand the transportation system or carry out new economic stimulus programs:
"Value for money" was a common theme observed during the scan. The agencies frequently used benefit-cost analysis to evaluate projects and programs and demonstrate the effectiveness of their investments.
The agencies visited clearly embraced performance management as the system for delivering results and documenting accountability.
The scan team found that in general, national governments have steadily reduced the number of measures and targets required of transportation agencies and moved toward fewer, broader, more policy-oriented goals.
"Do it with people, not to them" was both a direct quote and a common sentiment heard from transportation officials during the scan. From Sweden to New Zealand, transportation officials advised that carrots versus sticks, incentives versus penalties, and dialogue versus dictates were preferred in the intergovernmental management of performance.
Typical was the Swedish example, in which the central government expressed several broad goals for the transportation department. From those, the Swedish Road Administration negotiated a comprehensive set of about 300 performance measures it developed for both internal and external reporting. Hard targets for those measures were negotiated between the agency and the ministry. For instance, its target to reduce traffic fatalities by another 20 in 2009 was a short-term milestone toward a national vision of zero fatalities. It also targeted a department reduction in greenhouse gas emissions of 70,000 tons in 2009 as a short-term milestone toward reducing national carbon emissions. Both targets were negotiated with the central government. If the agency fails to achieve the targets, the performance is noted and a determination is made on what different tactics are needed to achieve the goal.
In short, the measures are benchmarks for continuous improvement and dialogue, rather than milestones for penalties.
The transportation agencies tended to speak to the public in broad, outcome-based terms, such as "the journey home" or "support for the journey," instead of technical terms.
In many cases, the data are packaged for a lay reader. Rather than extensive technical and financial data, public reporting focused more on outcomes of more immediate relevance and clarity to the typical user.
Many officials noted that successful performance management systems are long-term, iterative processes that require a commitment of funding and staff resources. All of the agencies had dedicated staff to collect and report performance data. All reported that measures evolve and often take significant effort to develop. The British spent 4 years refining their reliability measure and already are exploring a next generation of measure that is more understandable to the public.
Important outcomes that are difficult to measure in the United States were equally elusive in the agencies studied. Such measures as transportation's effect on the economy, travel time reliability, and transportation's effect on the environment were not easily captured by the visited agencies. All of the agencies expressed a desire for continued evaluation of ways to measure transportation's effects in these areas, particularly the economy.
All of the agencies officials support performance management, but they also spoke of a tendency of elected officials to emphasize short-term accomplishments in lieu of long-term goals. Several agency officials cautioned that while frequent budget reporting of results achieved short-term transparency, they feared that emphasis on "bean counting" skewed performance toward easily measured, short-term accomplishments. They advised that a better system was one that tracked accomplishment of long-term goals, which may be more ambiguous to measure but more important overall. Important issues such as the public's satisfaction with the journey, transportation's support of economic development, and transportation's link to environmental sustainability may be vitally important but difficult to measure in monthly increments.
Several agencies cited examples in which their performance reporting was used to criticize the agency, either in the media or in political debates. Several acknowledged that such criticism creates a desire to set easily achievable targets, hide or downplay problems, or play "metric games," which undermine the transparency and accountability of performance management.
While they had public reporting processes, all of the agencies had some form of candid, confidential reporting of results to central ministries. Great Britain used the Prime Minister’s Delivery Unit (PMDU) delivery assessment, an uncompromising, truthful assessment conducted every 6 months as a confidential tool. It was conducted as an evidenced self-assessment and action-setting exercise by government departments that was challenged and moderated by the PMDU. It drove the agency’s actions for the next 6 months. These confidential reporting processes allowed for candid discussions with central governments about agency performance, steps that may be needed to improve performance, and best practices.
The six agencies visited were in a state of transition—in terms of both their organization and their basic mission.
Three were in the process of reorganizing to merge the highway with state or regional transit agencies. The merger was driven by a national government desire to move away from a traditional highway-centric focus to a broader, more inclusive approach to surface transportation planning in highly congested urban areas.
"We are moving people, we are serving business, and we are moving freight. We are no longer in the business of just moving cars," said a New South Wales official. "We are no longer in the business of counting cars. It is about allocating road space."
"We are a travel agency. That is what we are involved with. It is not just the road," said a Swedish Road Administration official. "We are community builders."
"(We) had to do a cultural realignment to recognize that we are part of the solution for public transport," said a New South Wales highway official.
"The most important message was that we are the road authority, but we manage the transport network as one network that includes roads, buses, and trains. More and more, we are doing integration," said a VicRoads official in Melbourne. "From a road authority perspective, we can't build enough roads. If we did, it would not be a city anyone wants to live in. We need to manage the demand in travel."
The cause and effect of the agencies' performance management systems and their shift to holistic transportation agencies were not entirely clear. It appeared that the agencies' forecast of continuing degradation in travel time reliability pushed them beyond strategies of only expanding highways. The examined agencies placed a great emphasis on transit service, rail passenger service, land use integration, and moving people and freight as well as vehicles.
While the agencies displayed a strong commitment to transit, passenger rail, and urban land use integration, they also had ambitious highway corridor programs. All of the agencies retained a strong commitment to rural connectivity that relied heavily on highways. The Swedish and New Zealand governments emphasized their environmental commitments, but they also had programs to improve national highway corridors. The Australian states all maintained rural connectivity as a basic goal of their transportation programs. Although the urban systems emphasized ambitious transit and rail programs, the agencies also retained a strong highway component, particularly as it relates to national corridors and rural access.
The agencies visited displayed a strong commitment to addressing climate change and sustainability. However, none had adopted, nor had imposed on them, requirements to reduce vehicle-miles of travel.
The agencies were particularly emphatic about documenting safety results.
Several agencies displayed a progressive attitude toward highway operations, spurred by their efforts to meet reliability goals. The British, in particular, had invested considerable effort in measuring reliability on high-volume national routes. All of the agencies reported that their reliability measures were still evolving and they were not entirely satisfied with their measurement tools. However, it was clear that the more urbanized agencies in the United Kingdom, Australia, and Sweden were investing considerable effort in measuring real-time highway, transit, and rail operations to improve travel time reliability, enhance transportation choices, and reduce greenhouse gas emissions.
A consistent finding throughout the study was that although hard-and-fast performance targets were waning, the steady, long-term practice of benchmarking to broad government goals tended to spur innovative solutions to major transportation challenges, such as improving operations and travel time reliability.
The scan provided considerable insight into the evolution of performance management among nations that have practiced it for at least a decade. Their systems have matured and evolved in ways that provide lessons for the United States. The scan also validated the use of performance management as an effective means to translate broad government goals into meaningful agency practice. The performance management systems observed abroad provided transparency and accountability to transportation programs, while also allowing flexibility to meet local needs.
The officials offered the scan team advice in several key areas of performance management. The following outlines their advice and the scan team’s conclusions:
|<< Previous||Next >>|