CONTENTS

Table of Contents
Introduction

1. Descriptive Statistics in Evaluation
2. Subgroup Analysis
3. Variable - Are They Related?
4. Correlation
5. The t-Test of Difference Between Means

Supplements
Resources
Glossary

Wading Through the Data Swamp:
Program Evaluation 201

Module Content Objectives Background Variables - Are They Related? Hypotheses Research Questions Levels of Measurement What Is Chi-square? Contingency Tables Marijuana Use Chi-Square Calculations Continued for Marijuana Use (steps 7-10) Alcohol Use Chi-Square Calculations for Alcohol Use Continued (steps 7-10) Chi-Square Applet Lessons Learned: What Jack Should Have Asked His Evaluator What Did Jack Learn About His Program From These Analyses? How Can Jack Use This Statistical Information To Improve His Program? How Can Jack Use This Statistical Information To Better Promote His Program? Module 3 Quiz

Alcohol Use

Chi-Square Calculations for Alcohol Use (steps 1-6)

Now let's take a look at the contingency table for alcohol use and work through the same steps.

Step 1. State your hypotheses.

Here are the hypotheses that relate to alcohol use:

H_{0 (null)}: There is no difference between the participants' and comparison group's use of alcohol at posttest.

H_{i (research)}: There is a difference between the participants' and the comparison group's use of alcohol at posttest.

Step 2. Collapse your data.

Kids who reported 0 times = nonusers
Kids who reported >1 time = users

Step 3. Insert the collapsed data into your contingency table.

	Participants N=50	Comparisons N=50
Number and proportion of kids who drank in past month	Cell A 20 (40%)	Cell B 38 (76%)
Number and proportion of kids who did not drink in past month	Cell C 30 (60%)	Cell D 12 (24%)

Step 4. Add up totals for each row and column.

	Participants N=50	Comparisons N=50	Total
Number and proportion of kids who drank in past month	Cell A 20 (40%)	Cell B 38 (76%)
Number and proportion of kids who did not drink in past month	Cell C 30 (60%)	Cell D 12 (24%)
Total

Step 5. Compare the frequencies.

This table tells a different story for alcohol use than the one for marijuana. It looks like there is a real difference between the program participants and the comparison group.

Twenty kids from the participant group and 38 kids from the comparison group drank alcohol. From this, it would seem obvious that more kids from the comparison group were drinking alcohol.

However, we cannot just eyeball the numbers. We need to use statistics to make sure these differences are "real." For contingency tables, we always use a chi-square statistic to do this. chi-square can help us determine if differences are statistically significant.

This is how we determine that the differences are not due to chance. Jack's evaluator used the chi-square test to determine statistical significance. He used this particular statistic because both the independent variable (program participation) and dependent variable (alcohol use) are nominal-level variables, in this case "yes" or "no."

To determine if differences are statistically significant, we need to go through several more steps. Hang in there.

Step 6. Select a probability level.

In the social sciences, findings with more than 5 percent likelihood of happening by chance are generally considered to be "not significant." We express this likelihood as p = 0.05. This means that we are 95 percent sure that the differences are "real." (Think of it this way: 100 percent certainty - 5 percent chance = 95 percent certainty.)