CONTENTS

Table of Contents
Introduction

1. Descriptive Statistics in Evaluation
2. Subgroup Analysis
3. Variable - Are They Related?
4. Correlation
5. The t-Test of Difference Between Means

Supplements
Resources
Glossary

Wading Through the Data Swamp:
Program Evaluation 201

Module Content Overview Objectives Background Comparing Means What Is the t-Test For? What Does the t-Test Assume? Paired t-Test or Independent t-Test? Mathematical Symbols Steps for Calculating a Paired Sample t-Test for Participant Pre- and Posttests Results for the Paired t-Test Why This Information Does Not Tell Us All We Need To Know Independent Samples t-Test for Pretest Data Steps for Calculating an Independent Samples t-Test for Pretest Data So, Are the Participants and Comparisons Comparable Regarding Inhalant Use? Independent Samples t-Test for Posttest Data Steps for Calculating an Independent Samples t-Test for Posttest Data The Results for the Independent t-Test at Posttest Practice with t-Tests Lessons Learned: What Jack Should Have Asked His Evaluator What Did Jack Learn About His Program From These Analyses? Other Statistical Possibilities How Can Jack Use This Statistical Information To Improve His Program? How Can Jack Use This Information To Better Promote His Program? Module 5 Quiz

Comparing Means

Comparing means by merely looking at them gives us SOME information about the success of a program. But, as you read in the previous modules, it does not provide us with the whole picture. This is because two means may:

• Vary from one year to the next (kids change over time)
• Differ before and after a program (ideally, due to the given intervention, but possibly due to other factors)

Even though the means may be different, this difference may not be statistically significant. For this reason, a statistical test, called a t-test, has been created to determine whether the difference between means is meaningful.

The best way to get a sense of how a t-test works is to do one. Since we're comparing means, we'll need to look at both groups. The evaluator only gave the means for participants , so we'll need to calculate the means for the comparison group.

Inhalant Use Scores for Comparison and Participant Groups (Number of Days Used in the Past 30)

Participant Group			Comparison Group
#	Pretest(x)	Posttest(x)	#	Pretest(x)	Posttest(x)

Here are the results.

Participant Group		Comparison Group
Pretest	Posttest	Pretest	Posttest
1.64	2.32	1.54	3.5

Now that we have the means, let's look at a graph of them. We'll start with the participants.

This figure shows that the difference between the two means is less than a day (2.32 -1.64 = 0.68). Is looking at the difference of 0.68 enough for us to claim that the means are really that different?

Of course NOT!

The reason is that in the world of numbers, perceptions are not enough. (Scientists and statisticians need to make a living too.) The only way to be sure about what your numbers tell you is to run statistical tests.