Reliability

KEY:

ds = dataset you are currently using.

DV = dependent variable of interest

IV = independent variable of interest

XYXY = dummy name for a variable, matrix, or data frame into which you are moving information.

Cronbach's Alpha

Step 1: Transpose Your Data

In all likelihood your rows will need to be turned into columns. Before proceeding, please see the section on data transposition.

Step 2: Replace null cells with 0

In some cases (e.g. time-limited tests) some participants will not complete as many test items as do others. In these cases, you may be left with blank spaces. If your desire is to treat these as incorrect responses, then you will need to replace the blanks with zeros.

This command will search dataset XYXY for any cases in which na=true. In those cases the cell will be set to zero.

_____________________________

ds[is.na(ds)] <- 0

_____________________________

Step 3: Perform analysis

You will need the command "alpha" from the psych package.

[-1] excludes the first column (subject number) from the results, which are saved in XYXY.alpha.

_____________________________

library(psych)

XYXY.alpha <- alpha(ds[-1])

_____________________________

Split Half - Sperman Brown with Odd/Even Split

Step 1: Data Split

In this procedure you will first assign a value of 1 to a new column (here named "Half").

Next the provided R procedure will scan the rows of the column and change 1 to 2, whenever the previous row was a 1. It restarts whenever subject number changes (here named "Subject").

_____________________________

ds$Half = 1

for (i in 2:(length(ds$Half))){
  if ((ds$Subject[i] == ds$Subject[i-1]) &
     (ds$Half[i] == ds$Half[i-1])) {
  ds$Half[i] <- (ds$Half[i-1] +1)
  }}

_____________________________

Step 2: Aggregate scores for each half.

For each participant create an aggregate score of performance on odd trials and on even trials. Note that "FUN = mean" assumes that the goal is to generate mean scores. Adjust as needed.

_____________________________

XYXY.Split <- aggregate(list(DVname = ds$DV),
                           by=list(Subject = ds$Subject, Half = IV$Half),
                           FUN = mean, na.rm = TRUE)

_____________________________

Next you will need to transpose the data, such that odd and even rows become separate columns.

This will require the "dcast" function from the reshape2 package.

_____________________________

library(reshape2)


XYXY.Split2 <- dcast(XYXY.Split, XYXY$Subject ~ XYXY$Half,
                        value.var = "DVname")

_____________________________

You can give these columns names, such as...

_____________________________

names(XYXY.Split2) <- c("Subject", "Odd", "Even")

_____________________________

Step 3: Calculate reliability

First, generate the correlation between the columns.

_____________________________

XYXY.r <- cor(XYXY.Split2$Odd, XYXY.Split2$Even)

_____________________________

Perform the Spearman Brown correction.

_____________________________

(2 * XYXY.r) / (1 + XYXY.r)

_____________________________