Chapter 6: Key Lessons Learned

At a time when the United States works to define a Federal-State-regional-local framework for transportation performance management, the international examples hold many clear lessons. The performance management systems of the studied agencies showed clear linkage between government expenditures and transportation agency results. The agencies demonstrated cost-effectiveness and continued improvement. They displayed responsiveness to emerging social concerns, such as climate change and urban sprawl. They clearly established the condition of their assets and the future consequences of current investments. The overall impression the scan team gathered from meetings with six leading performance management agencies is that similar performance practices could be implemented effectively in the United States.

Throughout the interviews, international transportation officials offered similar advice based on more than a decade of performance management experience. As a result, many had a long-term perspective to share on how attempts at performance management tend to begin, why they succeed, how they fail, and how they evolve. The following four major strategies summarize the strategies deployed abroad. The broad strategies are followed by 16 specific tactics that the scan team recommends for consideration in the United States.

Strategies for Implementation

"Do It With People, Not To Them"

None of the officials the scan team met with recommended mandating performance targets and imposing penalties if agencies fail to meet them. The officials had years of experience meeting performance goals set by central governments and ensuring that local governments achieve performance goals set by state and national governments. The agencies had the perspective of being the receiver of performance expectations from higher levels of government and the evaluators of performance by local or regional governments. Officials from Sweden, Great Britain, Australia, and New Zealand all described a similar approach. They urged a collaborative goal-and target-setting process. Many relied on negotiated service agreements in which different levels of government negotiate goals and targets. Negotiation allowed implementing agencies to help select targets and priorities that make sense in their local contexts.

Great Britain's evolution in national performance management was particularly enlightening. The central government initially developed a broad range of more than 2,000 metrics in its desire to ensure performance by its national agencies, regional organizations, and local governments. For more than a decade, national, regional, and local governments struggled with the burden of reporting. Gradually, the measures were reduced. Now local and regional governments report on a much smaller number of measures, and they decide what targets to meet for each measure. The new approach is intended to provide uniform national reporting for trend monitoring, but it also allows local governments to pick the focus areas important to their communities. Instead of a regulatory and punitive approach to measuring performance, the agencies described a supportive and collaborative one. Clearly, expectations were set by the central government for the state transportation agency or by the state agency for local agencies. However, the expectations were negotiated and failure to meet targets brought collaboration, not penalization. The relationships between levels of government were much more like a coach-and-player relationship than an umpire-player relationship. The joint setting of targets and tracking of trajectory to target was intended to create a shared sense of purpose and collaboration between levels of government. Metrics were used to track performance over time, not as short-term means to penalize a lack of a performance.

When performance targets were not met, the central agency was more likely to resort to engagement, training, peer review, and root-cause analysis with the agency that failed to meet a target. In only limited cases did one level of government sanction another for failure to meet an imposed transportation performance target.

The lack of sanctions in their performance management systems should not be construed as a lack of substance. To the contrary, the long-term focus on process improvement, performance reporting, and accountability in all of the agencies visited led to continual improvement in performance. What drove that performance was seldom fear of a penalty and more often a desire to achieve a higher level of service.

Focus on Long-Term Improvement, Not Short-Term Targets

As long ago as 1991 in their book Reinventing Government, Osborne and Gaebler warned about common shortcomings with performance measurement. When agencies are rewarded or penalized based on narrow performance targets, it is possible to skew logical behavior so that the agencies achieve the targets, even if the achievement creates other inefficiencies or costs. Agencies may be tempted to "cream" or skim off the top easy accomplishments to achieve numeric targets while avoiding more difficult, but possibly more important, activities. Measures must evolve, because they seldom are perfect. Qualitative analysis must be conducted along with quantitative analysis, because numbers alone seldom capture all aspects of performance, Osborne and Gaebler wrote.

The advice to not focus only on short-term metrics was confirmed in the scan, even if the agencies visited used different language to express it. All of them emphasized that the advantage of performance management was the tracking of long-term trends and the seeking of continuous improvement. Short-term targets are suspect, was the advice of many. Specific, short-term targets are suspect because data are often unreliable. Outside circumstances such as weather and economic crisis can influence short-term performance and cause targets to be missed through no fault of the agency. Most important outcomes can be achieved only over a number of years, not in a month or quarter. Major accomplishments, such as improving systemwide pavement conditions, reducing congestion, lowering crash rates, and integrating transportation and land use, cannot be achieved in uniform quarterly increments. However, they can be achieved over many years of steady focus, collaboration, and effort. During those years, progress will ebb and flow and targets will occasionally be missed. However, the most important objective is to achieve long-term improvement. Tracking long-term performance serves two important ends, which cannot be addressed through measuring only short-term metrics. First, focusing on long-term trends tends to minimize disruptive overreaction to fluctuations in performance caused by poor data or outside circumstances. Second, it encourages investments that provide the greatest long-term payoff, even if they do not produce the greatest short-term metrics. This focus on the best long-term investment is particularly important to managing infrastructure assets over their entire life cycle.

At all of the agencies visited, officials emphasized that the advantage of performance management was that it led to long-term, continuous improvement. The iterative setting of goals, measuring of performance, analysis of results, and readjustment of effort were the major benefits they emphasized. A focus on short-term targets can lead to "metric games," which result in the apparent achievement of short-term targets without regard for whether long-term performance has actually improved. Only after measuring for a number of years can agencies truly understand if they improved transportation system performance. "You can't invent this kind of scheme overnight. It has taken us 10 years," advised one British transportation official.

Focus on Outcomes, Not Process

The agencies the scan team visited appeared to focus more on outcomes than processes. In Great Britain, the focus on reliability led to innovative operations strategies that caused HA and police departments to reappraise their normal approach to alleviating congestion and reducing crashes. In Sweden, the highway agency collaborated with the postal service to have postal workers plow snow between rural mailboxes in the far north. In Queensland, the road authorities intentionally blurred funding sources and jurisdictional boundaries to encourage collaboration among state and local governments on maintenance of rural roads. In New South Wales, producing safer young drivers was valued more than quickly processing new drivers' licenses. In three agencies, barriers between the highway and transit organizations were broken down to reduce traditional transportation silos. These examples illustrate what appeared to be a significant focus on achieving important outcomes. It was not common to hear the agencies speak about adherence to process.

Related to the focus on outcomes was the expression of government goals for transportation. In the agencies visited, the outcomes they sought to achieve derived from clearly articulated central government goals for transportation. The states and nations had a statutory process in which the central government was required to express the broad goals and outcomes it desired from the transportation agencies—national and state agencies as well as local and regional agencies. These government outcomes did not include narrow performance targets. Rather, they expressed broad policy aspirations. The agencies, in turn, translated those policy aspirations into clear goals, measures, and targets, which they then reported. The systems observed abroad did result in transportation agency activities driven largely by a need to achieve the central government's desired outcomes. However, the central government did not appear to require a detailed or prescriptive planning process to achieve those outcomes. Instead, the transportation agency negotiated the process, measures, and targets to meet the government objectives. Continuous negotiation, dialogue, and regular reporting of results assured the central government that the transportation agencies achieved the outcomes it desired. It appeared that achieving outcomes was more important than adhering to process.

Don't Expect Clear Linkage of Performance and Budgets

Explicit linkages between agency performance and agency budget levels were not found. As one British official said, there was no mechanistic linkage between how well an agency performed—or failed to perform—and the amount of funding it received. Funding increases tended to be incremental and based on how much money the government had once other services were funded. Transportation funding increases or decreases were not observed to be tied to the agencies' ability or inability to achieve a performance target.

Most of the agencies visited abroad displayed sophisticated asset management systems that were linked to their performance management systems. As a result, the agencies could clearly demonstrate needed funding levels to sustain system conditions for bridges, pavements, and maintenance needs. However, the agencies repeatedly reported that demonstrated maintenance needs—and demonstrated agency performance—did not trigger funding increases commensurate with those needs.

These statements on the lack of linkage between budgets and performance should not be construed to mean that performance data were inconsequential in the budget process. Performance data clearly were important. Performance data were used to measure agency accomplishments, track trends, and hold the agency accountable. However, the data on system conditions and system performance did not appear—by themselves—to trigger higher levels of government expenditures to achieve specific performance targets, nor were penalties and rewards for performance common.

Where the scan team found a substantial increase in transportation investment was in relation to the achievement of broad, new national visions. The team found major funding programs in several instances. They were spurred by national objectives to stimulate the economy, improve important corridors, transform urban landscapes, and achieve ambitious national visions.

Potential U.S. Tactics for Implementation

As the United States considers a national performance management system for its transportation programs, the following tactics appear to hold great promise. They result from the best practices documented abroad and from the observations of the scanning team.

1. Articulate a limited number of high-level national transportation policy goals that are linked to a clear set of measures and targets.
2. Negotiate intergovernmental agreements on how State, regional, and local agencies will achieve national goals while translating them into State, regional, and local context and priorities.
3. Evaluate performance by tracking the measures and reporting them in clear language appropriate for the audience.
4. Collaborate with State, regional, and local agencies to achieve the targets by emphasizing incentives, training, and support—instead of penalties— as the preferred way to advance performance.
5. Perpetuate long-term improvement by understanding that the real value of performance management is the development of an improved decisionmaking and investment process, not the achievement of many arbitrary, short-term targets.
6. Improve the use of benefit-cost analysis and risk management to demonstrate value for money. Consider major project postconstruction evaluations to assess whether benefits included in the original benefit-cost assessments were realized.
7. Recognize that major national visions, not achievement of narrow targets, tend to generate new investment.
8. Convert long-term deferred maintenance needs into a long-term future liability calculation. This clearly links the budget to long-term system sustainability.
9. Demonstrate accountability for fund expenditures by producing annual performance reports on agency achievements.
10. Instead of using technical jargon, report results with language meaningful to the public, such as "the journey home" or "support for the journey." Detailed, technical terms should be used for internal reporting, but should be translated into understandable language for the public.
11. Collaborate with other cabinet agencies, including conducting periodic meetings with top leadership on cross-cutting issues such as economic development, public health, highway safety, and climate change.
12. Have a strong safety focus and document the results of safety measures, in addition to the usual measures of infrastructure condition, internal operations, transit, and ontime rail performance.
13. Focus on desired outcomes for travel time reliability that lead to expanded strategies for highway operations.
14. Learn from international examples of addressing climate change that rely on improving vehicles, fuels, and modal choice, but do not mandate reductions in travel or mobility.
15. Provide resources to enable high-quality data tracking, analysis, and reporting capabilities that allow for the use of performance data in decisionmaking.
16. Recognize that performance management is not a black box or simplistic solution. It is a culture to grow in the agency as an important consideration in the decisionmaking and investment process.