RStudio Workshop Series: Paired Samples T-Test

So today, we're going to be doing the RStudio workshop series lesson #7: paired samples t-test. So that means today, we're going to talk about the paired samples t-test (which assumes normality) and the Wilcoxon signed-ranked test (which does not assume normality). So let us begin.
So a paired samples t-test: we use a paired samples t-test to determine whether the means of two continuous variables – for example, before and after treatment – within the same group of participants differs. So if I was in the study, I would potentially do a test at timepoint 1, maybe I get a treatment maybe they give me a snack, and then I do another test at timepoint 2. So that would be when we would use a paired samples t-test design. We're looking to measure something about me (maybe a happiness score) at two different times, for example, within the same person. This is a parametric test: this test assumes normality. So if we think about that normal distribution: highest amount of data in the middle, lowest amount of data at the ends, that typical bell-shaped curve; we need that kind of data for this test. And if you need a little bit of additional help running this kind of test, we've got some guides available for you to look at. I'm hoping this is the updated version of my notes because we have a typo here…today, we're doing the paired sample t-test and this one says independent. So I think it's still the same, but if there's an issue, I can close it and re-open it. 
So we're going to get started today by setting our working directory and opening our dataset. So what this means, is we're going to tell the computer: “where is the file we're looking to use today”. So we can go to the Session button, and click Set Working Directory, and Choose Directory. There are lots of different ways to set your working directory within RStudio, but this way is pretty easy. So if we click: Session > Set Working Directory > Choose Directory, that will open up our dialogue box to show you where that file might be stored on your computer. So you have to download it from the e-mail I sent you and put it somewhere on your computer so you can use it today. So here I'm in: Documents > Classes and Workshops > Lindsay's workshops > MICRO WORKSHOPS > RStudio. I don't see anything in my File Explorer here, but I know it's the correct location so I can say “Open”. What this will do, is it will open the file – let me be more precise – this will set the working directory to the location of the file. We're going to open the file in a second. So we want to copy and paste, from the bottom left in our console, we want to copy and paste this line of code and replace what is on line 23, because line 23 is MY working directory on MY computer. So if you're working on your own computer, you want to make sure you replace that line to point to where it is on YOUR computer. And we can see that this works a) because we have some blue text in our console in this bottom-left section; blue text normally means it worked okay. And b) in the bottom-right, in the Files [window], I can actually see the list of all of the different files that I have in this folder, including our paired samples t-test file. So I know we're in the right spot.
So we've set our working directory; we have told the computer: “we would like to work with stuff that is in this specific folder on this specific computer”. The next thing we have to do is actually open the file. So we have to give it the exact name of the file, and we're going to use the read.csv() function today because the file we're trying to open is a .csv file. So we're going to give it a name; I've called mine Fake_Data. So I have Fake_Data = read.csv(“Fake_Data.csv”). And you have to spell the file exactly as it is spelled in your folder, so capital F, with an under score, capital D, and the file extension. So if I highlight line 25 and I click the “Run” button, we get some blue text in the bottom-left down in our console. And we also, in the top-right in our global environment, now have open data. So it says Data: Fake_Data, 30 observations of 7 variables. I could actually click this if I wanted, in the global environment. If I click on it, it will open it like it's an Excel spreadsheet. And I can see I have a Gender column, Fake_Data1, Fake_Data2, Fake_Data3, Fake_Data4, Colour, and Group. So it looks like my file has opened properly, and we used read.csv() as a function to do that.
We are now ready to actually do some stuff with this file. So if we're doing a paired samples t-test, there are certain assumptions that must be met in order for the results of the test to be valid. So we can start looking at some of these assumptions. The first assumption is that our dependent variable must be continuous. To run a paired samples t-test, we need to have a continuous variable that is integer (<int>), double (<dbl>), or numeric (<num>); these are the data types within RStudio that we're looking for. And in this case, we're actually going to use them in two separate columns. So in Fake_Data, we can take a glimpse() of our data to see what data types we have, and whether we've got the right set-up for our columns. If you have not come to a workshop before, you will need to run line 30: install.packages(“tidyverse”). What this is going to do, is take the tidyverse package from the Internet and put it on your computer. I already have it on my computer, so I can skip line 30. But I do need to run line 31, which is: library(tidyverse). So I'm going to click “Run” on this, and this allows me to use tidyverse right now (i.e., use this package, which has a bunch of functions inside of it, so we can do some work with what we're trying to do with our dataset). So we get a bunch of output in our console in the bottom-left; it says I have a couple conflicts, but that's okay, it's not an error code. What I'm going to do next is run line 32, which is glimpse of Fake_Data. So glimpse(Fake_Data) , and if I run this, it shows me my data. So it says you ran glimpse of Fake_Data; it has 30 rows and 7 columns. Which is good; that matches what it shows in our global environment. And it uses the dollar sign ($) to tell us about the different columns, so $Gender is the Gender column, and it's listed as <int> or integer type. Today, we're actually going to be looking at Fake_Data1 and Fake_Data2; these are listed as <dbl>, which is RStudio’s way of saying it's double (i.e., it's got some decimals included. So we're doing okay, this means continuous data within RStudio. So we get a check mark, we have passed assumption one: our dependent variable is continuous.
Assumption two: our independent variable must be categorical with two related, matched groups. It's a paired t-test, you need two different observations for each person, and they have to be linked or paired. So if I was doing the fake study I was saying a minute ago: let's say I do some sort of happiness test at timepoint 1, they give me a cookie, and then I do a happiness test at timepoint 2. So I need two different columns of data about that happiness data. So each observation must have a score 1 and score 2; if I'm one person in the study, I need score 1 and score 2. We can look at our data set by using View(Fake_Data). And if I Run this, it opens it like an Excel spreadsheet, and I see Fake_Data1 and I see Fake_Data2. If we assume each row is an individual participant, we have paired data because each person has a Fake_Data1 score and a Fake_Data2 score. They've got two different things here.
So we get another check mark, we've passed that assumption; we have two unique but matched (or paired) groups: Fake_Data1 and Fake_Data2.
Assumption three: the dependent variable should be approximately normally distributed for the DIFFERENCE SCORE between the two groups. This might sound a little bit different than how we've checked normality in the past. Here, the important thing that you have to latch onto, is we're looking at the difference score. What that means, is we have to do a subtraction: we have to take – in this case, we're going to take – Fake_Data2 and then subtract Fake_Data1. You could do it the other way, that's totally fine as well; you could do Fake_Data1 subtract Fake_Data2. That's fine, that's still the same thing, you're doing a difference score. But we're going to take one and subtract the other, and then check normality on the difference score. To create this, we've got what's on line 49. So we're making a new column; the stuff on the left is saying make this as a new column. We've got Fake_Data, dollar sign ($) to say this is going to be a column in the data set, and we're just going to call it “diff” (diff for difference). Because this column does not already exist, that means this will create the column for us, and it will call it diff because the dollar sign indicator ($) again says this is going to be a column. We're going to set that equal to (=) Fake_Data$Fake_Data2 (so we're going to take Fake_Data2 as a column) and we're going to subtract Fake_Data$Fake_Data1. So we're going to take Fake_Data2 column and subtract Fake_Data1 column, and we're going to store that in what we call the “diff” column (i.e., the difference score column). We're doing a subtraction. So, if we click on line 49 [Fake_Data$diff = Fake_Data$Fake_Data2 – Fake_Data$Fake_Data1], and we click Run, we get some blue text in the bottom-left in the Console. This means that it probably worked. And we can check to see if it worked by running what's line 51: we can View() our fake dataset. So if we click Run, we now have this “diff” column. It's at the very end of the data set; whenever you create something new, it puts it at the end, but we have a diff column. And what that has done, is it has said: “Take Fake_Data2, subtract Fake_Data1”. So if we look, we've got 88.2 and we've got 93.6, let's say. So 93.6 - 88.2 is approximately 5.3 or 5.4. So it's done a difference score for us. And because when we assess normality for a paired sample t-test, we're looking at the difference score column, we're now going to check for normality in this new column that we've just created. All right, what does that actually look like? So we're going to run what we call a Shapiro-Wilk statistic. It's one way to check for normality. We're going to use Shapiro-Wilk because it's a statistic, it's going to give us a p-value. And we're using the Shapiro-Wilk statistic instead of the Kolmogorov–Smirnov statistic because we have a relatively small sample size; we have less than 50 observations. So we're going to run: shapiro.test(Fake_Data$diff). This will run the Shapiro Wilk statistic for our difference score to check normality. If we click Run here, we get some output in our Console; it says: Shapiro-Wilk normality test…data is Fake_Data$diff (we're using that difference score column). Our statistic is W = .96931 and our p-value = .5205. Because this is an assumption check, if our p-value is less than (<) .05, we would have a problem; we would have broken this assumption. Here, our p-value is greater than (>) .05, so we're okay, we have passed this assumption.
We're all right, we don't have to switch tests. If you failed this assumption, that might be when you switch to the Wilcoxon test instead. So we've passed our assumption of normality, that's number three. 
Next, we have assumption number four. I haven't given you any code here, but I've left you some comments. So for outliers, again, you're going to check on the DIFFERENCE SCORE between the groups. But different fields and different departments have different ways that they check for things like outliers, so I haven't left you any code here. There are different methods. You could do something like the mean and the standard deviation of the difference score column, and if a data point falls too far away, you would remove that data point as being an outlier. Some fields use box plots or histograms and they do visual inspection and say: “Does something look really far away from the rest of the data?”. So if you're doing this on your own dataset and you wanted to make sure you're doing it properly, you should check in with your research team to see how they identify and remove outliers just to make sure you're using what's common for your field. So again, I didn't leave any actual code here to check anything, but I've left you a note that you should still be checking this on the difference score column.
So those are our four assumptions for this specific test. We actually have passed our assumptions today. We didn't check outliers, but we checked normality and we passed, so because we've passed our assumptions, we can continue with the paired samples t-test. If we had failed assumptions…so let's say we failed outliers, maybe you remove the outliers, and then check the assumptions again; if you pass, you're good, you can use the paired sample t-test. If you were to remove outliers and you still fail normality, for example, that would be an indicator that you should maybe switch to the other test. You should switch to the non-parametric Wilcoxon test instead. But today, we passed everything we can actually run the paired sample t-test. Let's do that.
We're going to check to see whether the mean of Fake_Data1 differs from the mean of Fake_Data2. And we're going to use the t.test() function. And we're going to Run line 70 because this will show us in our Help window a little bit of more information about this specific function. So we can say ?t.test() and what this does, is down here, it will give us a little bit more information. So we don't have to go to a web browser to get this. It's just the RStudio documentation you can get from the Internet. But we can get it directly within our Help window here so we don't have to change windows.
So it says t.test() and then in squiggly brackets it says: {stats}. That means it's from the stats package, which we can actually find within tidyverse. So we already have this installed because we've installed tidyverse today. It says: “Students t-Test. Description. Performs one and two sample t-tests on vectors of data. Usage. t.test(x, …)”. That might seem a little confusing; here's a lot of different pieces that we could give this test. And if we scroll down, it tells us about all of these different arguments and some of the language is not super user-friendly. So let's work through this together and I'll show you what you need to run this test. So we can make this a little bit smaller. 
Okay, so we're going to run the paired sample t-test. You can do this a couple different ways. The easiest way that I'm going to show you today is t.test(Fake_Data$Fake_Data1, Fake_Data$Fake_Data2, paired = TRUE). 
What this is doing, is it's using the t.test() function. We know it's a function because it's some words and then a left round bracket ( “(“ ). We're going to compare the column Fake_Data1 to the column Fake_Data2, and we say “paired = TRUE” because each participant has a score for both Fake_Data1 and Fake_Data 2, so it's paired or matched data; “paired = TRUE” is how you tell this function it's a paired samples t-test. So if we highlight all of this and click Run, we will get some output in the bottom-left, in our Console. It gives you the line of code you ran and it says: paired t-test. Data is Fake_Data1 and Fake_Data2. It gives you a t-value. It gives you your degrees of freedom (df) and it gives you a p-value in scientific notation. So it says: 7.871e-15. That means our p-value is equal to: .000000000000007871. It's a fancy way of saying it (p) is less than (<) .05. This means that our groups are statistically significantly different: Fake_Data1 is different than Fake_Data2, and we're going to figure out which way that goes in a second. So we also get a 95% confidence interval here, and it tells you the mean difference between your two groups: the mean difference is 5.389584. This right now does not tell us which way it goes: is Fake_Data1 higher, [or] is Fake_Data2 higher? We don't know. So we've got our p-value, that's a good start. The next thing we might want is an effect size. For a paired samples t-test the effect size that we generally report is called Cohen's d, and it needs the “effectsize” package. So if you haven't installed this package before, you can run what's on line 77 [install.packages(“effectsize”)]. That will take it from the Internet and put it on your computer. I already have this installed, so I will skip to line 78 that says: “This is already on your computer, I would like to use it right now, I want to use this package”. So I will run: library(effectsize). And there's different ways to get the Cohen's d formula. We can use the same setup we did for the t-test: cohens_d(Fake_Data$Fake_Data1, Fake_Data$Fake_Data2, paired = TRUE. And if we Run this…it gave us a little bit of red text, but that doesn't look concerning to me. So that's okay, we can skip that, it's not an error code saying it didn't work. But it gives you a Cohen's d with a 95% confidence interval: our Cohen's d value is 2.65, and we've got a confidence interval if you want it.
So how do you actually write the results, and how do you know which group is higher or lower? We can do that next. So here's how you would write the results if you were doing it in a paper. And this is using some of the information that we just ran from Cohen's d, and some of the information from the paired t-test. So how you write this is: “t” (lowercase italicized t, because it's at t-test). You give it some brackets, and you give it your degrees of freedom, which is this value here, so 29 [ t(29) ] is equal to (=) your t-value. I'm dropping the negative because depending which thing you put where in the formula, one will be positive, one will be negative; it doesn't actually matter, so we can drop that negative and just say [ t(29) = ]14.511.
Comma you give it your p-value; You could write out all the decimals or you could give it scientific notation, depends on the researchers which way they like it, but you write out your p-value, so that's that point many, many zeros 7871 [ t(29) = 14.511, p = .000000000000007871]. And then you can give it d (lowercase italicize d for Cohen's d), and that's that 2.65; and same thing, we don't have to worry about that negative sign, we can drop the negative sign [t(29) = 14.511, p = .000000000000007871, d = 2.65]. That doesn't tell us which group is higher or lower. We can run line 84 and line 85. This is literally just the function mean(); it's mean(Fake_Data$Fake_Data1) and then the next one looks at mean(Fake_Data$Fake_Data2) instead. So if we run these ,we get a little bit more information. The mean of Fake_Data1 is 87.10212, and the mean of Fake_Data2 is 92.4917. So if we're looking back at that fake example from earlier, this means if I come into the lab and I do a happiness rating scale, I get at timepoint 1 maybe 87.1 is my score. They give me a cookie and suddenly I'm happier! So now my new score would be 92.5 if we round that.
So this means, with our p-value being less than (<) .05, we found a statistically significant difference between our two scores, and the second score is actually higher than the first score, and we know that because we can look at the means.
The last thing you probably want to do if you're running a paired samples t-test is you might want to visualize the data. We're not going to go too in-depth today about how to actually do plotting, because that's not the point of today; today is just learning how to do the test. But if you run line 90 and 91, this actually goes all the way down to 93 because we're using ggplot and that breaks it up in a few lines…but you can highlight this whole little chunk here and you can click Run. And what that will do is it will open up in your Plots window in the bottom-right, a graph of what we're looking at today. It's a little small, so we can click the “Zoom” button, and if I click Zoom, it will open this a little bit bigger. And we can see we've got two different groups: We've got Fake_Data1 on the left and Fake_Data2 on the right. We've got our mean of the groups here, and we can see that Fake_Data1 looks lower than Fake_Data2. Or you can say it the other way! Fake_Data 2 looks higher than Fake_Data1. So that's just a really simple, fast graph about the paired sample t-test we just ran, and it shows pretty clearly that there is a difference there. And that is how you run the paired samples t-test.