Appendix A: Amplifying Questions

It is becoming increasingly essential in the United States for transportation agencies at all levels of government to demonstrate credibility with elected officials and the public. The U.S. Congress is considering a performance-based transportation financing program that may require States and metropolitan planning organizations to document their accomplishments and results on a set of nationally and/ or regionally established goals. Although many transportation agencies use performance management (PM) programs, the programs and approaches can differ significantly. Also, in many cases, PM programs are not explicitly tied to national and State budgets or national strategic goals.

The purpose of this scan is to conduct indepth reviews of how transportation agencies in other countries apply transparent and accountable PM programs to budgets and budget requests at the national, state, provincial, metropolitan, or local levels. The scan also seeks to identify examples of how transportation budgets and programs directly link to accomplishment of national, provincial, and local strategic goals.

Performance measurement. For this study, the scan team used the U.S. Government Accountability Office (GAO) definition of performance measurement as "the ongoing monitoring and reporting of program accomplishments, particularly progress toward preestablished goals." The GAO defines a program as "any activity, project, function, or policy that has an identifiable purpose or set of objectives." A "performance measure" is defined by the Federal Highway Administration as "a qualitative or quantitative measure of outcomes, outputs, efficiency, or cost-effectiveness. In general, measures should be related to an organization's mission and programs, and should not merely measure one-time or short-term activities."

Performance management. The American Association of State Highway and Transportation Officials defines PM as an ongoing process that translates strategic goals into relevant and detailed measures and targets that, along with resources, are continuously monitored to ensure achievement of published institutional goals. Comprehensive performance management uses the definition just described in all key functions of a transport agency, including policy development and long-range planning; programming and budgeting; program, project, and service delivery; system operation; and monitoring and reporting of results in a variety of forms to a variety of audiences.

Strategic Approach

A major objective of this study is to examine examples in which national, state, or provincial strategic goals are translated into meaningful performance measures for the transportation agency. We want to examine practices in which the transportation agency uses those measures to document its achievement of society's transportation goals.

1. What were the strategic influences or specific catalyst that caused you to adopt performance management?
2. Is there alignment of strategic goals at the national, state, or provincial level down to the local level? If so, how does that work and how did your PM process help you?
3. At what level or multiple levels of government do transportation performance measurement and, more broadly, performance management occur: national, provincial, state, or local levels?
4. Has the process resulted in an improved connection between long-range transportation plans and the selection of individual projects?
5. Are there any performance measures in use that are based on national policies to affect growth in travel, such as reductions in vehicle-kilometers traveled, or achieve certain outcomes, such as reductions in greenhouse gas emissions or economic development?
6. How has the process affected the investment in different modes or in making tradeoffs between modes?

Setting Performance Goals, Measures, and Targets

Another important objective of this scan is to identify ways to establish effective and achievable performance levels based on input from the public, elected officials, and the business community. We seek examples of how to set performance goals at the proper level–not just easily achievable goals.

7. Performance measures, and targets set by your agency? What role do key stakeholders play in setting targets? Who are they?
8. How are measures influenced by the availability of data?
9. What considerations of scale–national, state, local, and project level–go into the choice of any particular performance metric?
10. Were the costs to achieve performance targets considered as part of the process to set targets? If so, how do you make financial tradeoffs between competing goals?
11. What timeframes do the performance targets typically cover? Are both long-term goals and short-term targets established?
12. How do you address significant differences between the conditions across the country or province, and how are these differences addressed in your performance measures and performance goals?
13. Do you emphasize outcomes, outputs, productivity, or satisfaction? Has this changed over time?
14. What organizational resources were required to implement your performance management process? Did you need to reorganize your organization or make other significant changes to accommodate performance management programs?
15. What types of functions are best suited to the use of performance measures and which are not? How did you decide?

Linking Performance, Budgets, and Results

Another important purpose of this scan is to find examples of tying performance and transparency to national and state, provincial, and metropolitan budgets. Many transportation officials in the United States believe it has become essential for their transportation agencies to demonstrate credibility with elected officials and the public, which may lead to sustained or increased funding for transportation. We also want to understand if budget or investment decisions between programs or geographic areas have been affected by the use of performance measures.

16. How have you used performance measures to explain financial needs to legislatures, administrators, the general public, and others to allow for the consideration of funding alternatives, etc.?
17. How are performance targets communicated to elected officials, the public, and the business community?
18. Have you used performance management in marketing campaigns to increase funding or investment in transportation? Were these successful?
19. What are the links between your agency's performance management system and your agency's budget?
20. Are measures and targets used to balance funding decisions for system preservation versus other important needs, such as system expansion or systems operations?
21. Are decisions to invest in one mode of transport versus another based on performance metrics or policy?
22. Do you approach decisions about system preservation with the same decisionmaking process you use for making decisions about system operation, expansion, or enhancement?
23. Has the use of performance measures changed the geographical or program distribution of funds?

Demonstrating Accountability

We are seeking ways transportation agencies can demonstrate good governance and accountability in meeting or exceeding performance expectations. We also want to understand how accountability is achieved.

24. Do your performance measures convince legislators of your efficiency and effectiveness
25. What tools have you found work best with your constituents and stakeholders to describe results?
26. What happens when goals are not met?
27. Have programs, projects, or activities been dropped as a result of indicators of poor performance or a lack of public need?
28. Are employees or organizations held accountable for meeting performance? If so, in what way?
29. When two or more transportation or other public agencies have a role in meeting performance targets, how is accountability established for each agency?