3.1 Linear model

The fitting model must accept at least an outcome vector y and a regressor matrix x:

fitFunLM = function(y, x){lm.fit(y = y, x = cbind(1, x))}

The predictive model must accept arguments mod (the fitted model) and x, the regressor matrix for a new set of observations.

predFunLM = function(mod, x) {cbind(1,x) %*% mod$coef}

Now that these functions have been defined, we apply the prediction model for leaf_8_width using the first 10 genes. Multithreading is used automatically using the BiocParallel package. Change the following setup depending on your system.

library(BiocParallel)
nCores = 2 # For cRAN build
register(MulticoreParam(nCores))

Now estimate the \(R^2\), also a rough estimate of the computation time is given. Remember to provide the cluster for multithreading.

R2lm = R2oosse(y = Brassica$Pheno$Leaf_8_width, x = Brassica$Expr[, 1:10], 
               fitFun = fitFunLM, predFun = predFunLM)

## Fitting and evaluating the model once took 0 seconds.
## You requested 200 repeats of 10-fold cross-validation with 10 cores, which is expected to last for roughly
## 2.05 seconds

Estimates and standard error of the different components are now available.

#R2
R2lm$R2

##        R2      R2SE 
## 0.4332216 0.1647254

#MSE
R2lm$MSE

##       MSE     MSESE 
## 3.2415675 0.7110744

#MST
R2lm$MST

##      MST    MSTSE 
## 5.719286 1.035600

Also confidence intervals can be constructed:

# R2
buildConfInt(R2lm)

##      2.5%     97.5% 
## 0.1103657 0.7560775

#MSE, 90% confidence interval
buildConfInt(R2lm, what = "MSE", conf = 0.9)

##       5%      95% 
## 2.071954 4.411181

By default, cross-validation (CV) is used to estimate the MSE, and nonparametric bootstrapping is used to estimate the correlation between MSE and MST estimators. Other choices can be supplied though, e.g. for bootstrap .632 estimation of the MSE and jackknife estimation of the correlation:

R2lm632jn = R2oosse(y = Brassica$Pheno$Leaf_8_width, x = Brassica$Expr[, 1:10], 
                    fitFun = fitFunLM, predFun = predFunLM, methodMSE = "bootstrap",
                    methodCor = "jackknife")

## Fitting and evaluating the model once took 0 seconds.
## You requested 200 .632 bootstrap instances with 10 cores, which is expected to last for roughly
## 2.56 seconds

3.2 Regularised linear model

For high-dimensional problems, such as the Brassica dataset, a regularised linear model is better suited to incorporate information for all genes. We use the cv.glmnet function from the glmnet package, which includes internal cross-validation for tuning the penalty parameter. Following custom function definitions are needed to fit in with the naming convention of the oosse package.

fitFunReg = function(y, x, ...) {cv.glmnet(y = y, x = x, ...)}
predFunReg = function(mod, x, ...){predict(mod, newx = x, ...)}

We adapt the parameter settings a bit to reduce computation time of the vignette.

if(require(glmnet)){
    R2pen = R2oosse(y = Brassica$Pheno$Leaf_8_width, x = Brassica$Expr[, seq_len(5e1)], #Subset genes for speed
                    nFolds = 4, cvReps = 1e2, nBootstrapsCor = 30,
                        fitFun = fitFunReg, predFun = predFunReg, alpha = 1)#Lasso model
    R2pen$R2
}

## Loading required package: glmnet

## Loading required package: Matrix

## Loaded glmnet 4.1-6

## Fitting and evaluating the model once took 0.17 seconds.
## You requested 100 repeats of 4-fold cross-validation with 10 cores, which is expected to last for roughly
## 29.41 seconds

##         R2       R2SE 
## 0.62688448 0.09036653

3.3 Random forest

As a final example we use a random forest as a prediction model. We use the implementation from the randomForest package.

if(require(randomForest)){
    fitFunrf = function(y, x, ...){randomForest(y = y, x, ...)}
    predFunrf = function(mod, x, ...){predict(mod, x, ...)}
    R2rf = R2oosse(y = Brassica$Pheno$Leaf_8_width, x = Brassica$Expr[, seq_len(5e1)],
                     nFolds = 4, cvReps = 1e2, nBootstrapsCor = 30,
                        fitFun = fitFunrf, predFun = predFunrf)
    R2rf$R2
}

## Loading required package: randomForest

## randomForest 4.7-1.1

## Type rfNews() to see new features/changes/bug fixes.

## Fitting and evaluating the model once took 0.08 seconds.
## You requested 100 repeats of 4-fold cross-validation with 10 cores, which is expected to last for roughly
## 13.59 seconds

##         R2       R2SE 
## 0.67076588 0.08850596

The \(R^2\) estimate is comparable to that of the penalised regression model.