CONTENTS

Table of Contents
Introduction

1. Evaluation and the Program Planning Process
2. Data Analysis, or "What Do You Mean by That?"
3. Special Challenges in Evaluating Prevention Programs

Supplements
Resources

Evaluation for the Unevaluated:
Program Evaluation 102

Validity and Reliability

Validity refers to whether you are measuring what you intend to measure. For example, which do you think would be a valid measure of your weight?

1. A bathroom scale
2. Your spouse
3. That guy at the carnival who gives you a stuffed bunny if he can't guess your weight within 3 pounds

Clearly, a scale is the most valid. Your spouse will lie to keep out of trouble. And the carnival guy always uses a scale to verify his guess.

Reliability refers to how much you can trust the measurements. For example, you are short and full figured. On Tuesday, your bathroom scale says you weigh 150 pounds. On Thursday, it says you weigh 180 pounds. Your bathroom scale has:

1. High reliability
2. Low reliability
3. A sick sense of humor

How does this work with prevention programs? Questions need to be worded to yield valid and reliable responses. For example, asking, "Do you have an alcohol problem?" is not a valid or reliable question, because the response is subjective. In addition, it could vary depending on the person's mood or circumstances at the time. Asking about consumption is a more valid and reliable way to measure alcohol problems.

For Jack's evaluation, participants were asked how many days during the past 30 they had used alcohol. Although some participants might estimate, number of days is an objective measure. They are not saying whether they have a problem, only that they used alcohol a certain number of days. Thus, it is a valid measure.