fcirt: Forced Choice in Item Response Theory

Naidan Tu

Overview

The fcirt package was developed to estimate forced choice models using Bayesian method. Specifically, the Multi-Unidimensional Pairwise Preference (MUPP) model is estimated using the R package rstan that utilizes the Hamiltonian Monte Carlo sampling algorithm. Four functions (i.e., fcirt( ), extract( ), information( ), and bayesplot( )) are provided for model estimation, results extraction, item and test information computation, and Bayesian diagnostic plottings, respectively. See below for function details.

Tutorial

Step 1: Input data

A randomly generated dataset is used as an example in this tutorial. The first input (fcirt.Data) is a dataset including responses from 5 respondents answering a 4 forced choice items (pairs) measuring 2 traits. Note that data is stored in a wide format.

ID item 1 item 2 item 3 item 4
1 1 0 0 0
2 0 1 0 1
3 0 1 1 1
4 1 0 1 1
5 1 1 1 0

The second input (pairmap) is a two-column matrix: the first column is the statement number for statement s; the second column is the statement number for statement t.

item statement 1 statement 2
1 1 2
2 3 4
3 5 6
4 7 8

The next part of the data is a column vector mapping each statement to each trait. For example, c(1, 1, 1, 2, 2, 2) means that the first 3 statements measure trait 1 and the last 3 statements measure trait 2.

row statement 1 statement 2 statement 3 statement 4 statement 5 statement 6 statement 7 statement 8
1 1 2 1 2 1 2 2 1

The last part of the data is a three-column matrix containing initial values for the three statement parameters (alpha, delta, tau) respectively. If using the direct MUPP estimation approach, 1 and -1 for alphas and taus are recommended and -1 or 1 for deltas are recommended depending on the signs of the statements. If using the two-step estimation approach, pre-estimated statement parameters are used as the initial values. The R package bmggum (Tu et al., 2021) can be used to estimate statement parameters for the two-step approach.

Statement Alpha Delta Tau
1 1 0 -1
2 1 1 -1
3 1 1 -1
4 1 0 -1
5 1 1 -1
6 1 1 -1
7 1 0 -1
8 1 0 -1

Step 2: Estimate using the function fcirt()

# Fit the MUPP model
#>mod <- fcirt(fcirt.Data=fcirt.Data, pairmap=pairmap, ind=ind, ParInits=ParInits, iter=100)
#>mod

The function fcirt() implements full Bayesian estimation of MUPP using rstan. The returned object stores information including the (1)stanfit object, (2)estimated statement parameters, (3)estimated person parameters, (4)response data, (5) the input pairmap, (6) the input initial values, and (7)the input row vector mapping each item to each trait. Below are a list of other arguments it contains, the default of which can be manually replaced:

Step 3: Extract the estimated results using the function extract()

# Extract theta estimates 
#>theta <- extract(x=mod, pars='theta')
# Turn theta estimates into p*trait matrix where p equals sample size and trait equals the number of latent traits
#>theta <- theta[,1]
# nrow=trait
#>theta <- matrix(theta, nrow=2)  
#>theta <- t(theta)
# theta estimates in p*trait matrix format
#>theta


# Extract tau estimates 
#>tau <- extract(x=mod, pars='tau')
#>tau <- tau[,1]
#>tau

The function extract() extracts fcirt estimation results.

Step 4: Plotting using the function bayesplot()

# Obtain density plots for all alphas. 
#>bayesplot(x=mod, pars='alpha', plot='density', inc_warmup=FALSE)
# Obtain the trace plots for all alphas.
#>bayesplot(x=mod, pars='alpha', plot='trace', inc_warmup=FALSE)

The function bayesplot() provides plots including density plots, trace plots, and auto-correlation plots to aid model convergence diagnosis. The smoothness of density plots, the stationary status of trace plots, and low degree of auto-correlation in auto-correlation plots all indicate good convergence.

Step 5: Obtain item and item information using the function information()

# Obtain item information for item 1-3.
#>OII <- information(x=mod, approach="direct", information="item", items=1:3)
#>OII
# Obtain test information.
#>OTI <- information(x=mod, approach="direct", information="test")
#>OTI