Paired Samples T-Test

[Lindsay Plater, PhD, Data Analyst II] 

So, what is a paired samples t-test?  

A paired samples t-test is used to determine whether the means of two continuous variables (for example, before and after a treatment) from the same group of participants differs.  

So here, we're looking for one row per participant. They've maybe got like a pre score and a post score. Or maybe you're looking at happiness, you gave some kind of drug treatment, so you're looking at the score before you give the drug and then after you gave the drug.  

But the trick here is that for each participant, the data have to be paired, there has to be some score at maybe timepoint 1 and then timepoint 2, or something like that where you can pair the data in some way. 

This is a parametric test, so it assumes normality. And what this means is that your data must follow the standard bell-shaped curve with the highest amount of data in the middle and lowering off to the ends.  

And if you're looking for help running a paired samples t-test, we have the University of Guelph SPSS LibGuide, the Laerd statistics guide, or the SPSS documentation.  

The trick with the paired samples t-test is it has multiple different names, so it might be called the dependent t-test, or repeated samples t-test, or a matched pairs t-test. So today, we're just going to call it paired samples t-test.  

What are the assumptions of a paired sample t-test? We have four of them. The first is that your dependent variable must be continuous. Your second assumption is the independent variable must be categorical with two paired groups or conditions. So, this is like that pre / post, it’s very common for a paired samples t-test. The third is that your dependent variable DIFFERENCE (so a difference score) between your groups or conditions is approximately normally distributed, and I'll show you how to calculate that difference score today. And the fourth is that you should have no significant outliers in the DIFFERENCE (so the different score) between your groups and conditions.  
So, we'll do all this in the next few slides, let's go.  

[Slide contains a screenshot of a table in SPSS within Data View. The table’s column headers are as follows: Gender, Fake_Data1, Fake_Data2, Fake_Data3, Fake_Data4, Colour, and Group.]  

Alright, checking our first assumption that your dependent variable must be continuous. Here we're looking at paired data, so we're going to look at both Fake_Data1 and Fake_Data2. 

[Fake_Data1 and Fake_Data2 columns are highlighted.]   

So, in this case, we're looking for each participant, they should have two observations, and it should be in two separate columns. And if we look in these columns, we've got some decimals, we've got some range of values.  

This data matches the assumption; so we've met the assumption, your dependent variable must be continuous.  

Our next assumption is that the independent variable must be categorical. It must have those buckets of information. Also, they have to contain two matched groups or conditions. So if we look for example across these rows, [first three columns (Gender, Fake_Data1, and Fake_Data2) of rows 1 and 16 are highlighted] we can look at participant one: this person identifies as male, they've got data in Fake_Data1 and they've got data in Fake_Data2, that's paired data, so we can use this. We've met the categorical independent variables assumption, because we've got participants who are either male or female.   

[Slide shows the table with the Transform menu open, and Compute Variable selected.] 

Okay, our next assumption, normality and outliers. We can check these at the same time, so we're going to do both of these in the same dialog box in SPSS.  

The trick with the paired samples t-test version of normality and outliers, is your looking at the DIFFERENCE SCORE. So, you're going to have to do an extra calculation before you can do this. So, what we're going to do before we even check normality and outliers, is we have to calculate the difference scores. And we do that by clicking Transform > Compute Variable.  

So again, for the paired sample t-test, when you're checking normality and outliers, there's a calculation you have to do first. You have to get a difference score. So, we're going to actually do a subtraction. So, if we click Transform > Compute Variable, that will open up the Compute Variable dialog box. 

[The Compute Variable dialog box in SPSS is shown, allowing users to create a new variable by defining a numeric expression. The Target Variable is named "diff_score", and the Numeric Expression calculates the difference between Fake_Data2 and Fake_Data1 (Fake_Data2 - Fake_Data1). The interface includes an arithmetic keypad, function groups (e.g., arithmetic, date functions), and options for case selection.] 

 And what we can do is we can write in the name for the test – or, not the test, pardon me – for the variable, so here I've called it something very exciting, I’ve called it “diff_score”, you can call yours whatever you like. But give it some meaningful name so you know what this is; so, I know that this is a different score because I've called it diff_score. And what you do, is you do a little bit of math! In the “Numeric Expression:” box, you're actually going to take your variables from the left side [box containing all of the variables from the table] and do a subtraction in the Numeric Expression box. 

So here I've got Fake_Data2, I've taken that and I've dragged it or clicked the blue arrow to put it in Numeric Expression. I've clicked the minus sign [from the arithmetic keypad], because this works like a little calculator. I've clicked the minus sign, and then I've taken Fake_Data1, and I've also dragged that into the equation. I've got Fake_Data2 minus Fake_Data1. Once you've got your expression ready to go, you can click “OK”. And the trick here is, it really doesn't matter which way you do this: you can either do (Fake_Data2 - Fake_Data1), or you can do (Fake_Data1 - Fake_Data2). The trick is it's just a difference score between the two of them, the order doesn't really matter.   

Okay. Once you have done that, you should now have a new column in your spreadsheet called diff_score.  

[Table with new column has the Analyze menu open and Descriptive Statistics selected. From the Descriptive Statistics sub-menu Explore is highlighted.] 

We can kind of see mine hidden behind the dialog box here. It's calculated a new variable in our spreadsheet that we can use to assess normality and outliers.  

And I think you've seen this before, but we can click: Analyze > Descriptive Statistics > Explore, and what we're going to do is take that difference score column that we just calculated (if you don't have a difference score column, you have to go and make one), but we're going to take that difference score column, and we're going to put that through the Explore procedure to check for normality and to check for outliers. So again, Analyze > Descriptive Statistics > Explore.   

[The Explore dialog box is shown, with diff_score added as the Dependent List variable. The interface includes options to add a Factor List or Label Cases by. There are buttons available for Statistics, Plots, Options, and Bootstrap, with Plots selected. A separate Explore: Plots window is open, where Boxplots are set to Factor levels together, and Descriptive is set to Histogram Normality plots with tests is also checked.] 

That will open up the Explore dialog box, and what you're going to do is you're going to take your difference score variable that you just created (so whatever you've called yours, ours is called diff_score), and you take it from the left side and you put it where it says “Dependent List:”. If your variables have all been set up properly within SPSS in Variable View, this should have a little yellow ruler, because a continuous variable minus another continuous variable should give you a continuous variable, and the yellow ruler signifies that this is continuous.  

We put difference score under Dependent List, and we click “Plots”. There's a few choices you're going to want to make here; under Descriptive you're going to uncheck Stem-and-leaf (we don't really do those much anymore), you’re going to check where it says Histogram because you want to do some visual inspection, and importantly, you're also going to click where it says “Normality plots with tests”. This one’s key, because this is going to give you a statistic. And when you've done all that you click Continue [in the Explore: Plots dialog]. That's all you need to do, so you can click “OK” [in the Explore dialog box] and that will run your normality and outlier checking in SPSS.  

If you’ve clicked those buttons, you will get some output. The first thing we're going to look at is normality, so if you scroll part way down your output file, you'll get a “Tests of Normality” table, and we can use this to check a statistic to see whether our difference score passes normality.  

[The Tests of Normality table in SPSS displays results from the Kolmogorov-Smirnov and Shapiro-Wilk tests. The row represents the variable diff_score, while the columns include test statistics, degrees of freedom (df), and significance values (Sig.) for each test. Additional notes mention the Lilliefors Significance Correction for the Kolmogorov-Smirnov test.] 

If you have more than fifty group – uh, fifty observations per group – you're going to look on the left-hand side where it says Kolmogorov-Smirnov. If you have fewer than fifty observations per group – or per, um, in your difference score column – you're going to look where it says Shapiro-Wilk statistic. We're going to be using the Shapiro-Wilk because we have 30 observations. And what you're looking for is in the “Sig.” column: if your p-value is less than (<) .05, you have failed the assumption of normality, which means you cannot use the paired samples t-test; it would be inappropriate to do so. Here we can actually see our Shapiro-Wilk statistic, we've passed normality because our p-value is greater than (>) .05 [the value shown is .521]. So, using a statistic, we've passed normality.  

We can also do visual inspection, so if you scroll a little bit lower you should also see a histogram.  

[The histogram displays the distribution of diff_score values, with Frequency on the y-axis and diff_score values on the x-axis. The diff_score values range from 2 to 10, and the frequency values from left to right are 1, 3, 4, 3, 7, 5, 6, 0, and 1.] 

What we're looking for here, is if you squint a little bit, this histogram should follow our approximate bell-shaped curve from before; so highest amount of data in the middle with lowest towards the ends. So, this one's not perfect, you can see that, yeah, most of the data is in the middle, but there's something kind of weird going on the right-hand side. But if you squint a bit, this looks like it passes normality, it's approximately the right shape, and in conjunction with the Shapiro-Wilk statistic, you're okay here.  

One last piece of visual inspection before we move on. We can also look at our Q-Q plot. 

[The Normal Q-Q Plot of diff_score compares the observed values (x-axis) to the expected normal values (y-axis). The data points, represented as red dots, align closely with the black diagonal reference line.] 

So, we have a line going across the page and we've got a bunch of data points (yours are probably blue, I made mine red for the presentation), but you've got a bunch of data points and you're looking to see: are those points close to or touching the line? If yes, you've passed normality. If no, if a bunch of the points are pulling away from the line, you've probably failed normality. So here we've passed our assumption of normality. Excellent.  

The next thing we can do, is in the same output file, we should also have a boxplot to help us look for outliers. And again, this is on our difference score.  

[The boxplot displays the distribution of diff_score, with a red rectangular box representing the interquartile range (IQR). The median is positioned slightly above the center of the box. Two whiskers extend from the box to the minimum and maximum values, indicating the full range of the data. There are no apparent outliers. The diff_score values range from approximately 2 to 10, with the middle 50% of the data concentrated between about 4 and 7.] 

Our box pot is another way to check for – visually – whether we've got some outliers going on. We've got a median (the black line in the middle), our interquartile range, and our whiskers. And what we're doing is we're looking to see, do we have any data points outside of those whiskers? They'll be identified as either a circle or a star with a number; the number next to those circles and stars indicate which row of your data set might be an outlier. The circle being a regular outlier, the star being an extreme outlier. Here, we don't have any outliers identified using the boxplot method. There are other ways to do outlier checking in SPSS or other software options as well, but SPSS makes boxplots super easy, so it's just the fastest way to do it. In some fields, you might also do something like the mean and the standard deviation, but we're not covering that today.  

[Slide shows the table in Data View with the Analyze menu open and Compare Means and Proportions is selected. From the Compare Means and Proportions sub-menu Paired Sample T Test is highlighted.]   

All right. If you have passed all of your assumptions…today, we have passed all of our assumptions, so it's okay for us to do the paired samples t-test…if you have passed your assumptions, you can move on and you will click: Analyze > Compare Means and Proportions > Paired Samples T Test.  
If you have not passed all of your assumptions, you might need to do something to change that. So, if you have outliers, maybe you remove some outliers; you check your normality assumption again, for example. If you've failed normality, but you've got no outliers, maybe you have to switch to a different test.   

But if you've passed all your assumptions (reminder, always check your assumptions), you're going to click: Analyze > Compare Means and Proportions > Paired Samples T Test. 

That will open up our “Paired Samples T Test” dialog box.  

[Paired-Samples T Test dialog box where the left panel lists available variables, with Fake_Data1 and Fake_Data2 selected as a paired comparison. The right panel displays the paired variables table. The dialog includes an option to estimate effect sizes, which is selected, with choices for standardization methods such as the standard deviation of the difference, corrected standard deviation, or average of variances. Standard deviation of the difference is selected. Additional settings can be accessed via the Options and Bootstrap buttons.] 

A few things you have to click here. You're going to take your Fake_Data1 variable and put it where it says “Variable1”, and you're going to take your Fake_Data2 variable and you're going to place it where it says “Variable2” [in the Paired Variables table]. You want those to be on the same row; you can do multiple of these tests at the same time, so which test you're comparing have to be on the same row. Here, we're using row “1”. If you also wanted to compare Fake_Data1 versus Fake_Data3, you could put those then on row “2” and it would run both tests for you. And here as well, it doesn't matter whether you do 1 and 2 or switch them and say 2 and 1, the order doesn't really matter.   

If you have that all set up, you're going to click OK. It's pretty easy. 

And you'll get a fairly large output with multiple different tables here.  

[Paired-Samples T Test output in SPSS. The left panel shows the Output Navigator. The main panel presents the following tables: Paired Samples Statistics, Paired Samples Correlations, Paired Samples Test, and Paired Samples Effect Sizes.] 

The first thing you're going to see under your T-test table is “Paired Samples Statistic” [table].  
This gives you your mean, your N (or number of observations), your standard deviation, your standard error of the mean: some standard descriptive statistics of what's going on within these two variables.  

The next you'll see the “Paired Samples Correlations” table. You don't really have to do much there; I don't normally look at that one.  

The thing you care a little bit more about is the third table, the “Paired Samples Test” table. It's got a bunch of different information here, it's essentially taking your two variables and saying: “In general, is one of our variables higher or lower than the other?” If you're looking for both higher and lower, or if you don't know which direction you might find a difference in, you're going to look where it says “Two-sided p” [a column header], this is pretty standard. If you had a previous hypothesis about what might be going on, and you think that, for example, one is going to be higher than two, you might do a “One-sided” t-test. But generally, if you're not sure, using a two-sided p, it's pretty standard. So, this table gives you stuff like your t value, your degrees of freedom (df), and both a two- and a one-sided test.  

All of that information will be useful if you're writing it in a paper. But here, what we're looking at is our two-sided p in our Significance section. And what this tells us, is if p is less than (<) .05, we have found statistical significance; and what that means is that one of our variables is higher or lower than the other. So here, our p-value is less than (<) .001, we found statistical significance, and what we can do is we can actually look up at that first table I talked about [the Paired Samples Statistics table] and look at the means to see which one is higher than the other. The p-value tells us they are different, the means can tell us in which direction. Our Fake_Data1 variable has a mean of 87.1, and our Fake_Data2 variable has a mean of 92.5 (if you round that). So, this means Fake_Data2 is higher than Fake_Data1, and that's backed up with our p-value.  

 There's one other important piece of information that you might want, it's in our last table, the “Paired Samples Effect Sizes” table. What we're looking at here is the Cohen’s d value [Point Estimate column header is highlighted]. Cohen’s d is a measure of effect size. How large is this effect?  

Alright, so that is how you run your paired samples t-test.