CONTENTS

Table of Contents
Introduction

1. Descriptive Statistics in Evaluation
2. Subgroup Analysis
3. Variable - Are They Related?
4. Correlation
5. The t-Test of Difference Between Means

Supplements
Resources
Glossary

Wading Through the Data Swamp:
Program Evaluation 201

The t-Test of Differences Between Means:
Independent Samples t-Test for Pretest Data

Step 2: Choose a significance level

Remember that statistical reasoning does not provide certain answers, such as yes or no. Rather, it gives answers in terms of probabilities.

If we want to reject the null hypothesis and support the research hypothesis, we have to select a probability that a true null hypothesis is being rejected. This is the probability that the mean difference we observe between participants and nonparticipants is due to chance and not to the Cool After School program.

Convention in the social sciences dictates the use of 0.05 and 0.01 levels of significance. A 0.05 level of significance means a 5 percent probability that the differences occurred by chance. A 0.01 level means a 1 percent chance. Usually these two levels are accompanied by the letter p, which means probability.

The lower the p-values, the more confident you can be that the effect you are observing is due to the After School program. Once we reject the null hypothesis, then the research hypothesis holds. We can then conclude that there is a difference between the means of the two groups. For our example, we are selecting a 0.05 level of significance.