Appendix B: Amplifying Questions

Overview of Performance Measures

• Please briefly describe the process of transportation system performance measurement used by your agency. Why did your organization begin using performance measures? How long has system performance measurement been used in your agency? What do you consider innovative about the process you are using for measuring transportation system performance?
• What performance measures are being used by your organization to monitor transportation system performance? Why were these measures selected?
• For what purposes have these measures been used (e.g., reporting to the public, reporting to oversight groups, internal management, prioritizing programs and projects, etc.)? How would you describe the benefits your agency gets from using performance measures?
• Which measures have you found to be the most important to your agency for the purposes identified in question 3 above? Which measures have not proved to be very successful for the purposes identified in question 3 above?
• Some agencies use measures simply to monitor transportation system performance over time. Others establish desired targets or goals for each measurement category (e.g., for safety, a national goal of reducing fatalities by x percent) . Do you establish targets or goals for any of your system performance categories? If so, who established these targets and why were these target values chosen?
• Does your agency use measures that describe the linkage between transportation system performance and the economy, (e.g., contribution of transportation to gross domestic product)? Are measures defined for any other non-transportation areas, such as transportation';s impact on environmental quality or international trade?.
• To what extent is performance measurement common in other agencies in your government? If other agencies use performance measurement, do those agencies that have some policy linkage to transportation (for example, agencies relating to safety and enforcement, trade, environmental quality) use transportation-related performance measures?
• If your agency is focused on only one part of the transportation system ( for example, the road network), are you aware of other agencies that use measures to monitor the performance of other parts of the transportation system, such as airports, ports, rail services, urban transit, etc.?
• If you represent a national organization, do you have agencies at the provincial/ state or metropolitan levels that also use performance measures? If so, to what extent are these measures similar or different from the measures you use?
• Do you believe that the use of performance measures has measurably changed how your agency does business? If so, which measures have been most important to your agency? Do you think that the monitoring of a selected number of measures could appreciably affect transportation system performance over the long term?
• Given the importance of transportation to society and to individuals, are you using (or thinking of developing) performance measures that track "how much transportation costs?" To individuals? To communities? To different sectors such as the freight sector? Do you make a distinction (or envision making one) between private costs to the individual (e.g., vehicle, fuel, insurance, etc.) and public costs (e.g., infrastructure, control systems, and maintenance)? Do you track the share of various users' contributions to the costs of transportation (e.g., the revenues generated for transportation from trucking fees) ? To the share of economic benefit they derive from investment in the transportation system?
• In the United States, we often measure the costs of congestion as the value of time lost. Do you measure congestion costs? Do you use, or are you considering, system performance measures that go beyond congestion measures, for example, measures of urban mobility, accessibility, and system reliability?
• How are transportation costs tracked or considered in assessments of economic productivity and economic health in your country, state/province, or metropolitan area, if at all?
• To what extent is a "customer focus" included in your performance measures? How do you define your customers? What strategies or techniques are used to collect data on customer satisfaction or dissatisfaction with transportation system performance?
• From your experience, what are the key lessons in using performance measures in your agency's planning and decision-making processes? What has been your greatest success?
• From your experience, what are the key barriers to effective use of performance measures in your agency's planning and organizational decision-making processes?
• From what you know about how transportation system performance measurement is done in the United States, what do you consider to be the best parts of our approach toward performance measurement? What do you do better than the United States?

System Performance Measurement and Decision-making

• If your agency has a strategic plan, business plan, or core business strategy, to what extent are your system performance measures integrated into this plan? To what extent have performance measures actually affected investment decisions? How do you make sure that the agency's decisions and actions reflect the desired performance outcomes?
• Do you use performance measures to determine the overall effectiveness of investment in the transportation system? If so, to what extent do you believe such performance measures are meaningful or useful to your agency? To important stakeholders? To top decision-making officials outside your agency?
• If targets or performance goals are used as part of your agency's performance measurement (e.g., a maximum level of crashes on the road system), how are these targets or goals chosen? Who decides what these values should be?
• Are you using any methodology that is based on a "benefit/cost" to choose investment strategies among different modes of transportation?
• Does your performance measurement system allow for a direct comparison between investment for new infrastructure versus rehabilitation and operations-oriented investment?
• Does your jurisdiction use an asset management approach to capital planning and investment decision-making? If so, what measures are used to track infrastructure condition? Do you track "system needs" over time? If so, how do you forecast future demands for transportation services?
• Do you track how transportation investment is made (e.g., how much is financed through debt as contrasted to other forms of investment)? How much is borne by user costs versus costs of general government? Have you noticed a change in the mix of financing over time? Does your agency have performance targets for cash balances, revenues, expenditures, etc.?
• Have you faced a situation where two or more performance measures might have been incompatible from the perspective of agency decision-making (e.g., a safety performance measure that results in slowing traffic that might be in conflict with a performance measure oriented to improving vehicular flow)?
• To what extent are decisions relating to human and capital resource allocation in your agency tied to desired levels of system performance as specified by your performance measures? For example, if infrastructure condition is an important performance measure, do you think a higher level of staff resource has been dedicated to system maintenance because of this measure?

Specific Types of Performance Measures

Safety

• What are the types of measures used to monitor transportation system safety? Why have you defined these measures as you have (e.g., some countries use fatality rates while others simply use number of fatalities)? Do you have different measures as a function of geographic scale (e.g., national versus state/provincial versus metropolitan), or by level of congestion?
• Which of the measures identified in question 1 have really made a difference in the way your agency makes investment decisions?
• Improving highway safety requires many different types of strategies ( e.g., actions relating to engineering, education, enforcement, and emergency services) . In most countries, this means that many different organizations are important participants in the road safety program. Are there other agencies (e.g., police, civil aviation, maritime security, etc.) that use safety-related transportation system performance measures? To what extent have government and non government organizations in your country/state/ province/region adopted a common set of safety-related performance measures?
• Are these performance measures primarily used for engineering decisions (e.g., what types of crashes do we need to focus on), or are they used by agency leaders and political decision makers (e.g., how much money do we need to devote to transportation system safety)? Are the performance measures used to benchmark your system's performance with that of other countries or other agencies?
• Has your agency or government established specific safety targets or goals? If so, what are they, and what has been your recent experience in meeting these targets?
• What data do you collect for the performance measures identified in question 1? Who collects this data? What type of data management system or information technology do you use to collect and manage safety-related data as it relates to system performance measurement? Do you use the Internet for reporting safety data?
• Are you tracking performance of the total transportation system or are you classifying your data by type of crash or geographic area? Do you have the capability to do both?
• To what extent is safety performance recorded and reported in relation to the cause of the incident (e.g., driver performance or pedestrian behavior, engineering design or other human factors issues)?
• Do you collect data on pedestrian and bicycle injuries and/ or fatalities? Do you collect data on freight-related injuries and/ or fatalities (e.g., truck crashes)? Does the data identify who or what caused the incident?
• Is your tracking of safety data used to predict what might happen in the future with respect to expected safety performance, given changing design and demographic characteristics? For example, is the changing age distribution of the population at all considered as a possible factor in future safety experience?
• Have you been able to demonstrate cause and effect between system investments and changes in system safety performance?
• To what extent has safety performance monitoring affected program decisions at the administrative or legislative level? To what extent has the use of performance measures led to improved safety on your transportation system? Have specific safety strategies or project designs been adopted by your agency because of safety performance measurement?
• From your experience, what are the key lessons in using safety performance measures in your agency's planning and decision-making processes? What has been your greatest success?
• From your experience, what are the key barriers to effective use of safety performance measures in your agency's planning and organizational decision-making processes?

System Optimization

• How do you measure desired system operational performance? For example, do you use some indicator of level of service such as travel speed, delay, maximum throughput, congested hours, etc?
• Have you developed a measure of transportation system reliability that responds to the needs and desires of system users (both freight and public users)? Reliability is defined as the degree to which travel time on a particular transportation facility or on a transportation system is similar over repeated trips, that is, the variance of travel time for repeated trips is quite small.
• If such measures are used, how useful are they in conveying to administrators and/or legislators in your jurisdiction the magnitude of the problems facing the transportation system?
• If some indicator of delay is one of the measures used, how is delay defined (e.g., as a matter of time or cost or both)#363;Is the delay for shipping, goods delivery, and other freight movement included in your system performance determination?
• Do you make a distinction in your performance measurement between delay that is caused by accidents and vehicle breakdowns (referred to as nonrecurring delay), and recurring delay due to high levels of demand and insufficient capacity?
• In several countries of the world, data collection is being conducted by both public agencies and private firms to provide better information to the users of the transportation system. To what extent are public and private organizations involved with providing travel information to travelers in your country?
• Do you have information that indicates the amount of improvement that will occur when a particular type of change is made to the transportation system (this could result from physical improvements or changes in pricing and demand management)? For example, if your country/province or state/ metropolitan area has implemented intelligent transportation system technologies, how are you tracking the performance and benefits of these investments?
• From your experience, what are the key lessons in using system optimization performance measures in your agency's planning and decision-making processes? What has been your greatest success?
• From your experience, what are the key barriers to effective use of system optimization performance measures in your agency's planning and organizational decision-making processes?

Transportation System Performance and Environmental Quality

• Do you measure the impacts of transportation system performance on the natural environment and on communities? For example, do you track the transportation system contribution to the following:
• To air quality such as greenhouse gases? By mode or region? Changes in fleet performance and mix (e.g., increasing use of alternative fueled vehicles)?
• To habitat and species preservation? Have performance measures begun to emerge from the COST Action 341 initiative of the European Community?
• To water quality? Do you have measures relating to the amount of impermeable surface associated with transportation facilities? If your agency deals with snow and ice removal, how do you monitor the environmental impacts of your removal strategies?
• To noise levels? Do you have measures that track the extent and severity of noise impacts on communities or on sensitive land uses?
• To environmental compensation (e.g., the number of wetlands replaced or restored due to transportation construction)?
• To land use? Is the relationship between transportation investment and urban form an important social, economic, or political issue in your jurisdiction?

Program Delivery and Accountability

• Do you monitor specific characteristics of your agency's program delivery (e.g., time for project development, project quality, project cost, use of consultants versus your own staff, etc.)? Do you have intermediate steps in the project development process that serve as benchmarks allowing you to monitor project progress? What are your greatest barriers to efficient program/project delivery?
• Have you changed agency project delivery procedures in response to monitoring of program progress (e.g., a greater use of value engineering, increased use of design/build consultants, use of incentive/disincentive clauses, etc.)?
• How do you report your agency's financial performance (a public sector version of a balance sheet, cash flow tables, return on investment, etc.)?
• How do you convey to the public that you are accountable for delivering and maintaining a good transportation system? How do you evaluate whether your information-disseminating strategy is being successful (e.g., through the number of hits on your Web site)? If you have a customer-focused set of performance measures, do you think that the public has a more positive impression of your agency's activities because of the public reporting of system performance?