CONTENTS

Table of Contents
Introduction

1. Descriptive Statistics in Evaluation
2. Subgroup Analysis
3. Variable - Are They Related?
4. Correlation
5. The t-Test of Difference Between Means

Supplements
Resources
Glossary

Wading Through the Data Swamp:
Program Evaluation 201

Research Designs

Posttest only. This collects data on participants after they've been through the program. This is not a very good design because it provides limited information and no basis for comparison. In fact, CSAP doesn't allow its grantees to use posttest only designs. However, getting some data is better than having no data. Thus, if nothing else is available, posttest only can help.

Pretest-posttest design without a comparison group. This gives at least some idea about whether participants have changed at all for any reason, including the program. But keep in mind that any evaluation design without a comparison group is not strong enough to sort out the program's effects from other things that can cause change over time.

True experimental design (pretest-posttest design with a control group and random assignment of participants to either the program or the control group). This is the ideal option. If you can assign people randomly, you can compare two groups who are the same at the start. The only thing that differs is that one group gets the program. Thus, any differences afterward should be from the program.

Quasi-experimental design (pretest-posttest design with a comparison group but nonrandom assignment of participants to the program or control group). In the real world, it is usually not possible to assign people to program and control groups randomly. In the prevention field we tend to use "quasi-experimental" designs. This is similar to the true experimental design, but participants are not randomly assigned to the two groups. Instead, participants in the prevention program are compared to some other existing group that is similar. This may be another classroom or another school or kids from a similar neighborhood.