Reliability
KEY:
ds = dataset you are currently using.
​
DV = dependent variable of interest
​
IV = independent variable of interest
​
XYXY = dummy name for a variable, matrix, or data frame into which you are moving information.
Cronbach's Alpha
Step 1: Transpose Your Data
In all likelihood your rows will need to be turned into columns. Before proceeding, please see the section on data transposition.
Step 2: Replace null cells with 0
In some cases (e.g. time-limited tests) some participants will not complete as many test items as do others. In these cases, you may be left with blank spaces. If your desire is to treat these as incorrect responses, then you will need to replace the blanks with zeros.
​
This command will search dataset XYXY for any cases in which na=true. In those cases the cell will be set to zero.
_____________________________
ds[is.na(ds)] <- 0
_____________________________
Step 3: Perform analysis
You will need the command "alpha" from the psych package.
​
[-1] excludes the first column (subject number) from the results, which are saved in XYXY.alpha.
_____________________________
library(psych)
XYXY.alpha <- alpha(ds[-1])
_____________________________
​
Split Half - Sperman Brown with Odd/Even Split
Step 1: Data Split
In this procedure you will first assign a value of 1 to a new column (here named "Half").
​
Next the provided R procedure will scan the rows of the column and change 1 to 2, whenever the previous row was a 1. It restarts whenever subject number changes (here named "Subject").
​
_____________________________
ds$Half = 1
for (i in 2:(length(ds$Half))){
if ((ds$Subject[i] == ds$Subject[i-1]) &
(ds$Half[i] == ds$Half[i-1])) {
ds$Half[i] <- (ds$Half[i-1] +1)
}}
_____________________________
​
Step 2: Aggregate scores for each half.
For each participant create an aggregate score of performance on odd trials and on even trials. Note that "FUN = mean" assumes that the goal is to generate mean scores. Adjust as needed.
​
_____________________________
XYXY.Split <- aggregate(list(DVname = ds$DV),
by=list(Subject = ds$Subject, Half = IV$Half),
FUN = mean, na.rm = TRUE)
_____________________________
​
Next you will need to transpose the data, such that odd and even rows become separate columns.
​
This will require the "dcast" function from the reshape2 package.
​
_____________________________
library(reshape2)
XYXY.Split2 <- dcast(XYXY.Split, XYXY$Subject ~ XYXY$Half,
value.var = "DVname")
_____________________________
​
You can give these columns names, such as...
​
_____________________________
names(XYXY.Split2) <- c("Subject", "Odd", "Even")
_____________________________
​
​
Step 3: Calculate reliability
First, generate the correlation between the columns.
_____________________________
XYXY.r <- cor(XYXY.Split2$Odd, XYXY.Split2$Even)
_____________________________
​
Perform the Spearman Brown correction.
​
_____________________________
(2 * XYXY.r) / (1 + XYXY.r)
_____________________________
​
​