CONTENTS

Table of Contents
Introduction

1. Descriptive Statistics in Evaluation
2. Subgroup Analysis
3. Variable - Are They Related?
4. Correlation
5. The t-Test of Difference Between Means

Supplements
Resources
Glossary

Wading Through the Data Swamp:
Program Evaluation 201

The t-Test of Differences Between Means:
Paired Sample t-Test for Participant Pre- and Posttests

Step 2: Choose a significance level

The key to grasping this concept is understanding that statistical reasoning does not provide certain answers, such as yes or no. Rather, it gives answers in terms of probabilities.

If we want to reject the null hypothesis and support the research hypothesis, we have to select a probability that a true null hypothesis is being rejected. This is the margin of error that the differences we observe between participants and nonparticipants are due to chance and not to the program.

Let's assume that you select 5 percent as your margin of error. In plain English, this 5 percent indicates that out of 100 times, the difference we observed was due to chance only five times. In the social sciences, it is standard to use 0.05 and 0.01 as levels of significance. Usually these two levels are accompanied by the letter p, which means probability.

The lower the p-values, the more confident you can be that that the effect you are observing is due to the program.