CONTENTS

Table of Contents
Introduction

1. Descriptive Statistics in Evaluation
2. Subgroup Analysis
3. Variable - Are They Related?
4. Correlation
5. The t-Test of Difference Between Means

Supplements
Resources
Glossary

Wading Through the Data Swamp:
Program Evaluation 201

Module Content Objectives Background Variables - Are They Related? Hypotheses Research Questions Levels of Measurement What Is Chi-square? Contingency Tables Marijuana Use Chi-Square Calculations Continued for Marijuana Use (steps 7-10) Alcohol Use Chi-Square Calculations for Alcohol Use Continued (steps 7-10) Chi-Square Applet Lessons Learned: What Jack Should Have Asked His Evaluator What Did Jack Learn About His Program From These Analyses? How Can Jack Use This Statistical Information To Improve His Program? How Can Jack Use This Statistical Information To Better Promote His Program? Module 3 Quiz

Chi-Square Calculations Continued for Marijuana Use (steps 7-10)

Step 7. Calculate chi-square.

Yes, of course, there's a formula:

X² = ([f_o - f_e]² / f_e)

where

X² = chi-square

= summation f_o

= observed frequency
f_e = expected frequency

The formula looks complicated, but it isn't so hard. The steps are:

1. For each cell in your contingency table, subtract the expected frequency (what would be expected by chance) from the observed frequency (what actually happened).
2. Square the result.
3. Divide by the expected frequency.
4. Add up the resulting numbers.

O.K., so maybe it is tricky. Let's walk through the process step by step. We already know the observed frequencies of alcohol use (number of kids who drank and who did not drink). Next we need to calculate the expected frequencies.

To calculate the expected frequencies for all four cells-A, B, C, and D-we need to multiply the observed frequency totals. These are the column and row totals for the column and row in which the cell appears. Then we divide by the total number of kids. We'll go through these cell by cell.

Expected Frequency, Cell A: Participants Who Smoked Marijuana, Past 30 Days

Marijuana Use in Past Month	Participants	Comparison Group	Total
Yes	Cell A 20	20
No	28	30	58
Total		50	98

Column Total x Row Total = 48 x 40
Expected Frequency (f_e) Calculation = (48x40)/100 = 19.2

Expected Frequency, Cell B: Comparison Group Who Smoked Marijuana, Past 30 Days

Marijuana Use in Past Month	Participants	Comparison Group	Total
Yes	20	Cell B 20
No	28	30	58
Total	48		98

Column Total x Row Total = 50 x 40
Expected Frequency (f_e) Calculation = (50x40)/100 = 20

Expected Frequency, Cell C: Participants Who Did Not Smoke Marijuana, Past 30 Days

Marijuana Use in Past Month	Participants	Comparison Group	Total
Yes	20	20	40
No	Cell C 28	30
Total		50	98

Column Total x Row Total = 48 x 58
Expected Frequency (f_e) Calculation = (48x58)/100 = 27.8

Expected Frequency, Cell D: Comparison Group Who Did Not Smoke Marijuana, Past 30 Days

Marijuana Use in Past Month	Participants	Comparison Group	Total
Yes	20	20	40
No	28	Cell D 30
Total	48		98

Column Total x Row Total = 50 x 58
Expected Frequency (f_e) Calculation = (50x58)/100 = 29

Now we're ready to calculate the differences between observed frequencies and expected frequencies. Then we'll square them and divide by expected frequency. These calculations are shown in the table.

Cell	Observed Frequency (fo)	Expected Frequency (fe)	(fo - fe)	(fo - fe)2	(fo - fe)2 / fe
A	20	19.2	.8	.64	.64/19.2=.333
B	20	20	0	0	0/20=0
C	28	27.8	.2	.04	.04/27.8=.001
D	30	29	1	1	1/29=.034
Total					X2 = .368

So, our chi-square value is .368. Well, that and a dollar will buy us a cup of coffee. This does not really mean much to anyone without knowing whether the value is statistically significant. Let's see how we find that out.

Step 8: Calculate degrees of freedom.

Whether a particular chi-square value is statistically significant depends on something called degrees of freedom. Degrees of freedom depend on the number of categories for each variable. The formula for calculating it is: (Number of categories in the row variable - 1 ) x (Number of categories in column variable - 1). Using our marijuana use example above, we can plug in the numbers to find the degrees of freedom. Degrees of freedom = (2 categories in row variable - 1) x (2 categories in column variable - 1) = (2 - 1) x (2 - 1) = 1.

Step 9: Determine chi-square critical value.

To find out whether our chi-square value is significant, we need to check a table of critical values. This table shows the minimum chi-square value for various degrees of freedom. To be significant at a particular degree of freedom, the chi-square value must equal or exceed the critical value. The table shows that for 1 degree of freedom (which is what we always have in a simple 2x2 contingency table), we need a chi-square value of at least 3.84 to be significant at the .05 level.

Step 10. Determine if our chi-square value is greater than the critical value.

Our chi-square value for marijuana use is .368. Since .368 is less than the critical value (3.84), Jack's evaluator failed to reject the null hypothesis of no difference. He therefore concluded that there is not a difference between the participants' and the comparison group's use of marijuana. In other words, Jack's program did not appear to have an effect.