Conference Proceedings 26 - Transportation Research Board

200 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS AND AGENCY OPERATIONS• How good is good enough? Some data uses havefinancial effects for the public or private sector. Althoughmost DOTs continuously strive for betterdataquality, it is also advisable that they establishappropriate disclaimers.1. The quality of data made available to managers,decision makers, and the public continues to bea significant concern to most DOTs. The level of attentionfocused on data quality may be growing dueto the greater accessibility to data that has been madepossible by posting data on public agency web sites.Workshop participants cited examples of externaldata users finding different values and different formatsfor the same data in various places within theagency’s web site. In other cases, data made accessiblethrough electronic formats such as web sites maydiffer from traditionally published data sources. Althoughthere will always be the possibility that somedata sources are more recent than others or may beformatted for different purposes, there is a need toensure that the data users obtain are consistent acrossthe board with what is available from all sourceswithin the DOT.2. The currency of data is also a concern, partlydue to the increased accessibility to data via the weband the availability of more frequently updated informationmade possible by greater use of automateddata-collection and -reduction techniques. Problemswere reported, such as when two users tapped intothe same data source and obtained data from differenttime periods. DOTs need to establish guidelinesand protocols that will determine when and how frequentlytheir data sources are updated. Users whoaccess agency data need to know what period thedata represent. Agencies should establish a procedurefor periodically updating published data and ensuringthat users understand the time period representedby the latest update. Pennsylvania (PennDOT) hasaddressed this problem by restricting public access todata from the most recently completed and publishedcollection period. With this approach, a DOT mayupdate internal databases continuously but releasethe new data summaries only periodically (e.g.,monthly, quarterly) to reduce the proliferation ofconflicting data sets.• Some DOTs expressed the concern that the inherentinaccuracies and imperfections in their datalimit the useful application of performance measuresin critical decisions. We need to be careful to avoidcreating a false perception of perfection or greaterexpectations for data quality and accuracy than wecan deliver. For example, if specific project decisionswith clear financial or safety implications are basedtoo heavily on performance measures, the outcomecan be undesirable. Decision makers should use dataas indicators of conditions and performance trends,for example, rather than as yes-or-no decision points.The opinion was expressed that we should let peoplemake decisions, using indicators to guide them, butthat we should not absolve them of the responsibilitythat comes with the decision-making authority.3. There is concern that DOTs may be using relativelycoarse and imprecise data to drive highly specificdecisions, for example, to select one project ordesign solution from among several possibilities. Althoughoften there is no better method, decision makersneed to be made aware of the quality of the datathat they are presented. We should do a better job ofdefining the variability of a given set of performancedata or the range of valid estimates rather than providea single data point for a given measure. It shouldbe the role of the data community to ensure thatmanagers and decision makers understand these datacharacteristics and the consequences of data variability.It was suggested that this is something the datacommunity can actually do something about, eventhough we may not be able to change the way inwhich decision makers use the data in their decisionprocess.Customer PerspectivesMost agencies acknowledge the importance of understandingthe audience for performance data. Theaudience can vary greatly, from internal group managersand staff to executive management teams toelected decision makers, to private-sector systemusers (e.g., freight carriers, land developers), to themotoring public, and to the general public, including‘‘nonusers.’’ Some recurrent themes included thefollowing:1. Defining the audience for performance measurementis desirable and will help shape datacollectionand -reporting programs. It is not alwayseasy to agree on who the customers are for a particulardata product, however. Careful thought on thatpoint is needed before data are developed andreleased.2. Although market research is conducted bymany DOTs, it is not necessarily accepted by all managersor decision makers as accurate or representative.Due to a lack of standardized methods or establishedregular survey programs, DOTs have oftenfound it difficult to combine or even compare resultsfrom different surveys.