Wading Through the Data Swamp:
Program Evaluation 201

Module 2: Subgroup Analysis - Page 4 of 12

Missing Data

Uh oh, did you say 48 kids at posttest? We're MISSING SCORES? That's right. Two of the kids did not answer the posttest question about marijuana use. So now what?

The conservative approach would be to exclude those kids from the analysis of the data for the marijuana question. Exclude both the pretest scores for which they both answered 0 days and the missing posttest responses. Therefore, the number of kids (N) will change from 50 to 48. We must use the new N value when calculating our statistics.

Don't try to GUESS based on the participant's pretest score. This approach doesn't work if you are looking at outcome data. However, if you are missing descriptive data, such as age or grade level, it is quite common to plug in the mean.

A picture of an atom.

There are many other more sophisticated techniques that your evaluator may use to analyze data with missing scores. Here are a few:

Jack should consider himself very lucky! He only had a few missing scores. Missing data caused from attrition and loss to followup is very common. (Our case would be considered loss to followup rather than attrition. The participants just failed to answer one particular question. They answered other questions on the posttest and were in the program until the end.)

A picture of an atom.

Can't We Just Tie Them to Their Chairs? How To Deal With Attrition and Loss to Followup

If you've had problems with attrition, you'll want to look at ways to improve your program. To avoid data collection problems, it might help to collect data more frequently. Consider collecting at different time intervals between pre - and posttest. Then you'll be more likely to have data on folks who may not be around for the posttest.

Another option is to provide incentives, such as movie tickets or even cash, on data collection days.