Evaluation for the Unevaluated:
Program Evaluation 101

Supplements

Areas of Evaluator Expertise

Topic Criteria
Process Evaluation Expertise and experience in collecting information and evaluating the extent to which the process of the prevention activity exceeded, achieved, or fell short of expectations, and the implications for this process on outcomes.
Psychometrics/
Instrumentation		 Experience and expertise in the development and application of instruments that reliably collect valid information from individual sources.
Statistical Analyses/Modeling Experience and expertise in the selection, use, and interpretation of appropriate statistical techniques to analyze quantitative data. This must be more than the ability to "plug in" standard formulas or techniques and includes understanding of how unevaluated factors may influence results.
Stochastic Modeling Experience and expertise in the selection, use, and interpretation of models that organize randomly selected observations into a probability framework. This type of evaluation is useful when there are a large number of uncontrolled observations, such as nighttime single car crashes, a typical measure of the success of prevention of driving under the influence.
Interview Facilitation Experience and expertise in eliciting and interpreting valid and reliable information from the interview process. This may include key informant interviews, individual participant interviews, telephone and community intercepts, group sessions such as focus groups, and iterative processes such as the Delphi technique.
Impact Evaluation. Experience and expertise in identifying and measuring the probable impact of a program on larger community processes, such as reduction in alcohol-related car crashes among youth. In addition, impact evaluation must incorporate unplanned impacts, such as declining participation in a competing youth development activity as a result of recruitment of youth into the prevention effort.
Cost-Benefit/Cost-Effectiveness Evaluation Expertise and experience in determining the economic implications of the prevention effort.
Policy Analysis Expertise and experience in examining the effects of authoritative decisions affecting outcomes, including worksite policies, community policies, and government laws, regulations, and ordinances. These include both intended and unintended consequences and variation based on level of enforcement
Clinical Trials Experience and expertise in conducting clinical trials. This is a specific subarea of evaluation that, in most cases, may not be suitable for substance abuse prevention. Clinical trials are usually used to test medications and treatment protocols.
Time Series Expertise and experience in this specific subarea of evaluation. It replaces measurement of change against a comparison with measurement of change in an intervention population over time.

<< Previous Page