Conference Proceedings 26 - Transportation Research Board

MEASURING THAT WHICH CANNOT BE MEASURED 107needs to be greater participation by numerous disciplinesin defining terms and in designing measurementapproaches. This paper lays the groundworkfor such a multidisciplinary approach, but it cannotprovide the expertise needed to develop these validperformance measures.Second, although performance-based planning hasbeen discussed and debated in the transportation professionfor the past 10 years, there are still some disagreementsand discrepancies on how they are perceivedand applied. Regarding the concepts in thispaper, it is important to understand how performancemeasures are considered in the context oftransportation planning and how they can be usedfor decision making. The next section presents aframework that illustrates the integration of performancemeasures into transportation planning. At thispoint, the following points need to be made.There is a clear distinction between performancemeasures and the evaluation criteria used to analyzealternatives, although the literature and practice oftenblur the distinction. Clearly, an important relationshipexists between the two, but each plays a verydifferent role in the planning process. For example,‘‘number of jobs generated’’ might be an evaluationcriterion used to assess the impacts of a transportationproject. A more general measure of economicvitality would be the system performance measure.Or the ‘‘number of tons of a particular pollutant’’might be an evaluation criterion, with a more generalsystems measure being quality of public health. Manyof the more general measures are difficult to measure,perhaps even to define, often leading to surrogatemeasures such as economic costs.Another important point-of-departure issue is theuse of performance measures in the decision-makingprocess. The performance measures discussed in thispaper are not measures to compare the effectivenessof one program with that of another. They are notintended to be used in an audit of a particular program’sperformance. There has been much confusionabout the use of performance measures in such a role.Rather, the performance measures discussed in this paperare intended primarily to monitor the performanceover time of the transportation system and to relatethat performance to the decision-making process leadingto investments in that system. This necessarilyleads to a discussion of cause and effect that is thefundamental challenge in using outcome-oriented performancemeasures in transportation planning and decisionmaking. To use performance measures in sucha capacity, we need to understand system performancemeasurement from a broad perspective.Third, although performance measures aimed atmonitoring the operation of a transportation systemare fairly straightforward, those that focus onbroader societal outcomes face significant challengesin definition and application. The underlying theorythat links transportation system performance andthese outcomes needs to be well established and believableby decision makers. For the types of societaloutcomes of interest to this paper, such theories willmost likely relate to economics, ecology, earth sciences,and human behavior. More importantly, therelationship between transportation system operationand investment will have a time lag associated withthe eventual outcomes, thus adding to the complexityin establishing cause and effect. This time lag couldalso affect decision makers’ interest in the performancemeasure to begin with. It seems likely thatwhatever set of transportation performance measuresis considered for broader society outcomes, it willconsist of those transportation-related variables thatoccur early in the cause-and-effect cycle or precursorsto the eventual outcome of interest.PERSPECTIVES FROM OTHER FIELDSBefore discussing the application of performancemeasures for broader societal outcomes in transportation,it might be instructive to examine briefly theexperience in three important fields of study. The literaturein each is voluminous and cannot be repeatedhere in detail. However, putting the transportationexperience in the context of these other approachesto a similar challenge can help to better understandhow the linkage between transportation system performanceand societal outcomes could be approached.At least it will show that our profession isnot alone in facing these challenges. Three fields ofstudy are presented: water resources, ecology andsustainability, and economics.Water ResourcesThe field of water resources planning and engineeringhas often preceded transportation planning and engineeringin its development and application of stateof-the-artpractices and technical approaches. For example,multiobjective analysis was being used forwater resources planning long before it was developedin the transportation field. The American Societyof Civil Engineers (ASCE) took the lead in developingsustainability criteria for water resourcesystems and related the management of water resourcesto much broader societal issues, such as publichealth, economic development, and environmentalquality (1998). In its treatise on the subject, ASCE