CONTENTS

Table of Contents
Introduction

1. Descriptive Statistics in Evaluation
2. Subgroup Analysis
3. Variable - Are They Related?
4. Correlation
5. The t-Test of Difference Between Means

Supplements
Resources
Glossary

Wading Through the Data Swamp:
Program Evaluation 201

Module Content Objectives Background Variables - Are They Related? Hypotheses Research Questions Levels of Measurement What Is Chi-square? Contingency Tables Marijuana Use Chi-Square Calculations Continued for Marijuana Use (steps 7-10) Alcohol Use Chi-Square Calculations for Alcohol Use Continued (steps 7-10) Chi-Square Applet Lessons Learned: What Jack Should Have Asked His Evaluator What Did Jack Learn About His Program From These Analyses? How Can Jack Use This Statistical Information To Improve His Program? How Can Jack Use This Statistical Information To Better Promote His Program? Module 3 Quiz

Chi-Square Calculations for Alcohol Use Continued (steps 7-10)

Step 7. Calculate chi-square.

Here's the formula:

X² = ([f_o - f_e]² / f_e)

where

X² = chi-square

= summation

f_o

= observed frequency
f_e = expected frequency

The formula looks complicated, but it isn't so hard. The steps are:

1. For each cell in your contingency table, subtract the expected frequency (what would be expected by chance) from the observed frequency (what actually happened).
2. Square the result.
3. Divide by the expected frequency.
4. Add up the resulting numbers.

Let's walk through the process again step by step. We already know the observed frequencies of alcohol use (number of kids who drank and who did not drink). Next we need to calculate the expected frequencies.

To calculate the expected frequencies for all four cells-A, B, C, and D-we need to multiply the observed frequency totals. These are the column and row totals for the column and row in which the cell appears. Then we divide by the total number of kids. We'll go through these cell by cell.

Expected Frequency, Cell A: Participants Who Drank, Past 30 Days

Alcohol Use in Past Month	Participants	Comparison Group	Total
Yes	Cell A 20	38
No	30	12	42
Total		50	100

Column Total x Row Total = 50 x 58
Expected Frequency (f_e) Calculation = (50x58)/100 = 29

Expected Frequency, Cell B: Comparison Group Who Drank, Past 30 Days

Alcohol Use in Past Month	Participants	Comparison Group	Total
Yes	20	Cell B 38
No	30	12	42
Total	50		100

Column Total x Row Total = 50 x 58
Expected Frequency (f_e) Calculation = (50x58)/100 = 29

Expected Frequency, Cell C: Participants Who Did Not Drink, Past 30 Days

Alcohol Use in Past Month	Participants	Comparison Group	Total
Yes	20	38	58
No	Cell C 30	12
Total		50	100

Column Total x Row Total = 50 x 42
Expected Frequency (f_e) Calculation = (50x42)/100 = 21

Expected Frequency, Cell D: Comparison Group Who Did Not Drink, Past 30 Days

Alcohol Use in Past Month	Participants	Comparison Group	Total
Yes	20	38	58
No	30	Cell D 12
Total	50		100

Column Total x Row Total = 50 x 42
Expected Frequency (f_e) Calculation = (50x42)/100 = 21

Now we're ready to calculate the differences between observed frequencies and expected frequencies. Then we'll square them and divide by expected frequency. These calculations are shown in the table.

Cell	Observed Frequency (fo)	Expected Frequency (fe)	(fo - fe)	(fo - fe)2	(fo - fe)2 / fe
A	20	29	-9	81	81/29=2.8
B	38	29	9	81	81/29=2.8
C	30	21	9	81	81/21=3.9
D	12	21	-9	81	81/21=3.9
Total					X2 = 13.4

So, our chi-square value is 13.4. As you know, this does not really mean much to anyone without knowing whether the value is statistically significant. Let's see how we find that out.

Step 8: Calculate degrees of freedom.

Whether a particular chi-square value is statistically significant depends on something called degrees of freedom. Degrees of freedom depend on the number of categories for each variable. The formula for calculating it is:

(Number of categories in the row variable - 1 ) x (Number of categories in column variable -1)

Using our alcohol use example above, we can plug in the numbers to find the degrees of freedom.

Degrees of freedom = (2 categories in row variable -1) x (2 categories in column variable -1) = (2-1) x (2-1) = 1

Step 9: Determine chi-square critical value.

To find out whether our chi-square value is significant, we need to check a table of critical values. This table shows the minimum chi-square value for various degrees of freedom. To be significant at a particular degree of freedom, the chi-square value must equal or exceed the critical value.

The table shows that for 1 degree of freedom (which is what we always have in a simple 2x2 contingency table), we need a chi-square value of at least 3.84 to be significant at the .05 level.

Step 10. Determine if our chi-square value is greater than the critical value.

Our chi-square value for alcohol use is 13.4. Since 13.4 is larger than the critical value (3.84), Jack's evaluator rejected the null hypothesis of no difference. He therefore concluded that there is a difference between the participants' and the comparison group's use of alcohol.

From the information in the contingency table for alcohol use we could see that there were not as many kids drinking in the participant group as there were in the comparison group. After calculating the chi-square statistic, we now know that this difference is significant.