OTIS Evaluation Guide (PDF) - California Department of Public Health

March 2007
Introduction
To facilitate the design and implementation of sound evaluation efforts, the Tobacco Control Section (TCS) has
prepared the OTIS Evaluation Guide. TCS encourages local program evaluation through empowerment. In other
words, it is in the best interest of the program that the program staff understand the evaluation process, partner
with the evaluation consultant in designing the evaluation plan, and not turn the whole evaluation process over
to the evaluation consultant. The reason is simple: evaluation and intervention are integrated processes. By going
through the evaluation planning process with the evaluator, the program staff will be knowledgeable about the
evaluation, which leads to better program planning.
The OTIS Evaluation Guide (the Guide), formerly known as the Local Program Evaluation Guide, has been
revised. It contains information about evaluation, and is a “how-to” for entering your evaluation plan into OTIS.
Instructions were revised to take you through the process in a step-wise manner.
The Tobacco Control Section appreciate the feedback and comments received on the earlier versions of the
Guide. This version incorporates those comments to improve the user-friendliness.
The length of the Guide was decreased and the sample plans are now available on an enclosed compact disc
(CD). Each sample plan is a separate file and includes screenshots from OTIS containing evaluation plan details
and the evaluation plan narrative. At the end of each sample plan, we have included a narrative to explain how
and why the particular evaluation design was selected and how and why the particular data collection method
was used. Writing a narrative can help you think through all the details of an evaluation plan and come to an
understanding of the rationale behind your decision. An introduction on writing a narrative is included in the
“Developing the Evaluation Plan” chapter of the Guide.
Throughout the Guide, evaluation terms are designated with an icon ( ) alerting you to look for the definition
near the end of the Guide instead of breaking up the flow of discussion within the text.
Dividers/tabs are provided for each section of the Guide so that you can easily locate items, and checklists are
provided to facilitate organization of evaluation information before you enter it into OTIS.
We strongly encourage you to read the entire Guide, especially Chapter 1, “Perfecting Your Objective” and
Chapter 2, “Understanding How the Evaluation Plan is Entered into OTIS, Version 2.”
ix