traineR Package

PROMiDAT

2021-05-31

The traineR package seeks to unify the different ways of creating predictive models and their different predictive formats. It includes methods such as K-Nearest Neighbors, Decision Trees, ADA Boosting, Extreme Gradient Boosting, Random Forest, Neural Networks, Deep Learning, Support Vector Machines, Bayesian, Linear Discriminant Analysis, Quadratic Discriminant Analysis and Logical Regression.

The main idea of the package is that all predictions can be execute using a standard syntax, also that all predictive methods can be used in the same way by default, for example, that all packages are use classification in their default invocation and all methods use a formula to determine the predictor variables (independent variables) and the response variable.

Examples:

For the following examples we will use the Puromycin dataset:

conc rate state
0.02 76 treated
0.02 47 treated
0.06 97 treated
0.06 107 treated
0.11 123 treated
0.11 139 treated
0.22 159 treated
0.22 152 treated
0.56 191 treated
0.56 201 treated 
n <- seq_len(nrow(Puromycin))
.sample <- sample(n, length(n) * 0.7)
data.train <- Puromycin[.sample,]
data.test  <- Puromycin[-.sample,]

Logistic Regression

Modeling:

model <- train.glm(state~., data.train)
model
#> 
#> Call:  glm(formula = state ~ ., family = binomial, data = data.train)
#> 
#> Coefficients:
#> (Intercept)         conc         rate  
#>     2.29849      2.55722     -0.02734  
#> 
#> Degrees of Freedom: 15 Total (i.e. Null);  13 Residual
#> Null Deviance:       21.93 
#> Residual Deviance: 20.26     AIC: 26.26

Prediction as probability:

Note: the result is always a matrix.

prediction <- predict(model, data.test , type = "prob")
prediction
#>        treated untreated
#> [1,] 0.6863659 0.3136341
#> [2,] 0.8163233 0.1836767
#> [3,] 0.8538404 0.1461596
#> [4,] 0.2778341 0.7221659
#> [5,] 0.4612626 0.5387374
#> [6,] 0.6374826 0.3625174
#> [7,] 0.6432216 0.3567784

Prediction as classification:

Note: the result is always a factor.

prediction <- predict(model, data.test , type = "class")
prediction
#> [1] treated   treated   treated   untreated untreated treated   treated  
#> Levels: treated untreated

Confusion Matrix

mc <- confusion.matrix(data.test, prediction)
mc
#>            prediction
#> real        treated untreated
#>   treated         3         0
#>   untreated       2         2

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>            prediction
#> real        treated untreated
#>   treated         3         0
#>   untreated       2         2
#> 
#> Overall Accuracy: 0.7143
#> Overall Error:    0.2857
#> 
#> Category Accuracy:
#> 
#>       treated    untreated
#>      1.000000     0.500000

ADA Boosting

Modeling:

model <- train.ada(state~., data.train, iter = 200)
model
#> Call:
#> ada(state ~ ., data = data.train, iter = 200)
#> 
#> Loss: exponential Method: discrete   Iteration: 200 
#> 
#> Final Confusion Matrix for Data:
#>            Final Prediction
#> True value  treated
#>   treated         9
#>   untreated       7
#> 
#> Train Error: 0.438 
#> 
#> Out-Of-Bag Error:  0.438  iteration= 6 
#> 
#> Additional Estimates of number of iterations:
#> 
#> train.err1 train.kap1 
#>          1          1

Prediction as probability:

prediction <- predict(model, data.test , type = "prob")
prediction
#>      treated untreated
#> [1,]  0.5625    0.4375
#> [2,]  0.5625    0.4375
#> [3,]  0.5625    0.4375
#> [4,]  0.5625    0.4375
#> [5,]  0.5625    0.4375
#> [6,]  0.5625    0.4375
#> [7,]  0.5625    0.4375

Prediction as classification:

prediction <- predict(model, data.test , type = "class")
prediction
#> [1] treated treated treated treated treated treated treated
#> Levels: treated untreated

Confusion Matrix:

mc <- confusion.matrix(data.test, prediction)
mc
#>            prediction
#> real        treated untreated
#>   treated         3         0
#>   untreated       4         0

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>            prediction
#> real        treated untreated
#>   treated         3         0
#>   untreated       4         0
#> 
#> Overall Accuracy: 0.4286
#> Overall Error:    0.5714
#> 
#> Category Accuracy:
#> 
#>       treated    untreated
#>      1.000000     0.000000

For the following examples we will use the iris dataset:

Sepal.Length Sepal.Width Petal.Length Petal.Width Species
5.1 3.5 1.4 0.2 setosa
4.9 3.0 1.4 0.2 setosa
4.7 3.2 1.3 0.2 setosa
4.6 3.1 1.5 0.2 setosa
5.0 3.6 1.4 0.2 setosa
5.4 3.9 1.7 0.4 setosa
4.6 3.4 1.4 0.3 setosa
5.0 3.4 1.5 0.2 setosa
4.4 2.9 1.4 0.2 setosa
4.9 3.1 1.5 0.1 setosa 
data("iris")
n <- seq_len(nrow(iris))
.sample <- sample(n, length(n) * 0.75)
data.train <- iris[.sample,]
data.test <- iris[-.sample,]

Decision Trees

Modeling:

model <- train.rpart(Species~., data.train)
model
#> n= 112 
#> 
#> node), split, n, loss, yval, (yprob)
#>       * denotes terminal node
#> 
#> 1) root 112 72 setosa (0.35714286 0.30357143 0.33928571)  
#>   2) Petal.Length< 2.5 40  0 setosa (1.00000000 0.00000000 0.00000000) *
#>   3) Petal.Length>=2.5 72 34 virginica (0.00000000 0.47222222 0.52777778)  
#>     6) Petal.Width< 1.75 37  4 versicolor (0.00000000 0.89189189 0.10810811) *
#>     7) Petal.Width>=1.75 35  1 virginica (0.00000000 0.02857143 0.97142857) *

Prediction as probability:

prediction <- predict(model, data.test , type = "prob")
prediction
#>     setosa versicolor virginica
#> 1        1 0.00000000 0.0000000
#> 13       1 0.00000000 0.0000000
#> 16       1 0.00000000 0.0000000
#> 25       1 0.00000000 0.0000000
#> 28       1 0.00000000 0.0000000
#> 37       1 0.00000000 0.0000000
#> 40       1 0.00000000 0.0000000
#> 41       1 0.00000000 0.0000000
#> 45       1 0.00000000 0.0000000
#> 48       1 0.00000000 0.0000000
#> 51       0 0.89189189 0.1081081
#> 54       0 0.89189189 0.1081081
#> 62       0 0.89189189 0.1081081
#> 63       0 0.89189189 0.1081081
#> 64       0 0.89189189 0.1081081
#> 67       0 0.89189189 0.1081081
#> 68       0 0.89189189 0.1081081
#> 80       0 0.89189189 0.1081081
#> 82       0 0.89189189 0.1081081
#> 84       0 0.89189189 0.1081081
#> 85       0 0.89189189 0.1081081
#> 87       0 0.89189189 0.1081081
#> 89       0 0.89189189 0.1081081
#> 96       0 0.89189189 0.1081081
#> 98       0 0.89189189 0.1081081
#> 99       0 0.89189189 0.1081081
#> 102      0 0.02857143 0.9714286
#> 105      0 0.02857143 0.9714286
#> 115      0 0.02857143 0.9714286
#> 119      0 0.02857143 0.9714286
#> 122      0 0.02857143 0.9714286
#> 123      0 0.02857143 0.9714286
#> 130      0 0.89189189 0.1081081
#> 132      0 0.02857143 0.9714286
#> 133      0 0.02857143 0.9714286
#> 143      0 0.02857143 0.9714286
#> 144      0 0.02857143 0.9714286
#> 147      0 0.02857143 0.9714286

Prediction as classification:

prediction <- predict(model, data.test , type = "class")
prediction
#>  [1] setosa     setosa     setosa     setosa     setosa     setosa    
#>  [7] setosa     setosa     setosa     setosa     versicolor versicolor
#> [13] versicolor versicolor versicolor versicolor versicolor versicolor
#> [19] versicolor versicolor versicolor versicolor versicolor versicolor
#> [25] versicolor versicolor virginica  virginica  virginica  virginica 
#> [31] virginica  virginica  versicolor virginica  virginica  virginica 
#> [37] virginica  virginica 
#> Levels: setosa versicolor virginica

Confusion Matrix:

mc <- confusion.matrix(data.test, prediction)
mc
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         16         0
#>   virginica       0          1        11

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         16         0
#>   virginica       0          1        11
#> 
#> Overall Accuracy: 0.9737
#> Overall Error:    0.0263
#> 
#> Category Accuracy:
#> 
#>        setosa   versicolor    virginica
#>      1.000000     1.000000     0.916667

The model still supports the functions of the original package.

library(rpart.plot)
prp(model, extra = 104, branch.type = 2, 
    box.col = c("pink", "palegreen3", "cyan")[model$frame$yval])

Bayesian Method

Modeling:

model <- train.bayes(Species~., data.train)
model
#> 
#> Naive Bayes Classifier for Discrete Predictors
#> 
#> Call:
#> naiveBayes.default(x = X, y = Y, laplace = laplace)
#> 
#> A-priori probabilities:
#> Y
#>     setosa versicolor  virginica 
#>  0.3571429  0.3035714  0.3392857 
#> 
#> Conditional probabilities:
#>             Sepal.Length
#> Y                [,1]      [,2]
#>   setosa     4.985000 0.3591657
#>   versicolor 5.970588 0.5357122
#>   virginica  6.576316 0.5748956
#> 
#>             Sepal.Width
#> Y                [,1]      [,2]
#>   setosa     3.405000 0.3815958
#>   versicolor 2.764706 0.3227555
#>   virginica  3.000000 0.3162278
#> 
#>             Petal.Length
#> Y                [,1]      [,2]
#>   setosa     1.450000 0.1617215
#>   versicolor 4.285294 0.4513497
#>   virginica  5.507895 0.5042423
#> 
#>             Petal.Width
#> Y                [,1]      [,2]
#>   setosa     0.247500 0.1085747
#>   versicolor 1.344118 0.1909961
#>   virginica  2.015789 0.2899453

Prediction as probability:

prediction <- predict(model, data.test , type = "prob")
prediction
#>              setosa   versicolor    virginica
#>  [1,]  1.000000e+00 7.400932e-20 1.348206e-26
#>  [2,]  1.000000e+00 4.184387e-20 4.935605e-27
#>  [3,]  1.000000e+00 5.230150e-18 3.146672e-24
#>  [4,]  1.000000e+00 1.524602e-15 4.584825e-22
#>  [5,]  1.000000e+00 4.502547e-19 1.163711e-25
#>  [6,]  1.000000e+00 1.736558e-19 4.793530e-26
#>  [7,]  1.000000e+00 5.572007e-19 1.011311e-25
#>  [8,]  1.000000e+00 3.632105e-19 1.925656e-26
#>  [9,]  1.000000e+00 2.107296e-13 4.139813e-20
#> [10,]  1.000000e+00 1.071259e-19 5.345710e-27
#> [11,] 1.233478e-117 7.785007e-01 2.214993e-01
#> [12,]  8.276105e-76 9.999735e-01 2.646935e-05
#> [13,]  6.198830e-93 9.958479e-01 4.152140e-03
#> [14,]  1.002777e-66 9.999891e-01 1.094249e-05
#> [15,] 2.942568e-114 9.784854e-01 2.151463e-02
#> [16,] 1.807114e-106 9.894179e-01 1.058214e-02
#> [17,]  1.094481e-69 9.999583e-01 4.167560e-05
#> [18,]  1.538602e-45 9.999980e-01 2.046171e-06
#> [19,]  9.157277e-53 9.999979e-01 2.053595e-06
#> [20,] 8.534847e-146 5.564907e-01 4.435093e-01
#> [21,] 5.590446e-106 9.923191e-01 7.680851e-03
#> [22,] 1.065671e-120 7.657408e-01 2.342592e-01
#> [23,]  3.449775e-79 9.997358e-01 2.642305e-04
#> [24,]  3.617648e-80 9.997401e-01 2.598548e-04
#> [25,]  1.748106e-90 9.984078e-01 1.592201e-03
#> [26,]  1.317590e-31 9.999999e-01 1.317255e-07
#> [27,] 1.121179e-161 2.414187e-02 9.758581e-01
#> [28,] 6.179955e-231 1.712775e-07 9.999998e-01
#> [29,] 2.812357e-196 6.926661e-07 9.999993e-01
#> [30,]  0.000000e+00 1.576614e-12 1.000000e+00
#> [31,] 3.234047e-155 1.432243e-02 9.856776e-01
#> [32,] 6.697641e-296 1.959515e-09 1.000000e+00
#> [33,] 3.417133e-198 7.641346e-04 9.992359e-01
#> [34,] 5.500206e-271 1.110922e-09 1.000000e+00
#> [35,] 1.365737e-216 1.148364e-06 9.999989e-01
#> [36,] 1.121179e-161 2.414187e-02 9.758581e-01
#> [37,] 7.996794e-247 3.380471e-09 1.000000e+00
#> [38,] 6.813941e-158 2.673432e-02 9.732657e-01

Prediction as classification:

prediction <- predict(model, data.test , type = "class")
prediction
#>  [1] setosa     setosa     setosa     setosa     setosa     setosa    
#>  [7] setosa     setosa     setosa     setosa     versicolor versicolor
#> [13] versicolor versicolor versicolor versicolor versicolor versicolor
#> [19] versicolor versicolor versicolor versicolor versicolor versicolor
#> [25] versicolor versicolor virginica  virginica  virginica  virginica 
#> [31] virginica  virginica  virginica  virginica  virginica  virginica 
#> [37] virginica  virginica 
#> Levels: setosa versicolor virginica

Confusion Matrix:

mc <- confusion.matrix(data.test, prediction)
mc
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         16         0
#>   virginica       0          0        12

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         16         0
#>   virginica       0          0        12
#> 
#> Overall Accuracy: 1.0000
#> Overall Error:    0.0000
#> 
#> Category Accuracy:
#> 
#>        setosa   versicolor    virginica
#>      1.000000     1.000000     1.000000

Linear Discriminant Analysis

Modeling:

model <- train.lda(Species~., data.train)
model
#> Call:
#> lda(Species ~ ., data = data.train)
#> 
#> Prior probabilities of groups:
#>     setosa versicolor  virginica 
#>  0.3571429  0.3035714  0.3392857 
#> 
#> Group means:
#>            Sepal.Length Sepal.Width Petal.Length Petal.Width
#> setosa         4.985000    3.405000     1.450000    0.247500
#> versicolor     5.970588    2.764706     4.285294    1.344118
#> virginica      6.576316    3.000000     5.507895    2.015789
#> 
#> Coefficients of linear discriminants:
#>                    LD1        LD2
#> Sepal.Length  1.124712  0.5022016
#> Sepal.Width   1.421003 -2.4568406
#> Petal.Length -2.672942  0.5448688
#> Petal.Width  -2.604451 -2.4383341
#> 
#> Proportion of trace:
#>   LD1   LD2 
#> 0.993 0.007

Prediction as probability:

prediction <- predict(model, data.test , type = "prob")
prediction
#>           setosa   versicolor    virginica
#> 1   1.000000e+00 4.279885e-26 5.307384e-48
#> 13  1.000000e+00 4.688591e-22 2.619221e-43
#> 16  1.000000e+00 3.122947e-32 6.757260e-55
#> 25  1.000000e+00 7.147402e-18 8.032096e-37
#> 28  1.000000e+00 2.303507e-25 4.658727e-47
#> 37  1.000000e+00 3.183836e-29 1.865653e-52
#> 40  1.000000e+00 3.747875e-24 1.645712e-45
#> 41  1.000000e+00 9.095636e-26 2.656868e-47
#> 45  1.000000e+00 4.025196e-20 5.016116e-39
#> 48  1.000000e+00 1.677369e-21 5.934792e-42
#> 51  4.344294e-21 9.999623e-01 3.773942e-05
#> 54  2.453863e-25 9.997945e-01 2.054896e-04
#> 62  7.822648e-23 9.995399e-01 4.601252e-04
#> 63  1.700777e-20 9.999996e-01 3.632865e-07
#> 64  1.327702e-27 9.917009e-01 8.299119e-03
#> 67  6.386433e-28 9.585298e-01 4.147022e-02
#> 68  5.255689e-19 9.999993e-01 7.417301e-07
#> 80  5.915717e-13 1.000000e+00 3.949975e-09
#> 82  8.477967e-18 9.999999e-01 1.380672e-07
#> 84  8.319874e-38 4.957339e-02 9.504266e-01
#> 85  6.534139e-29 8.923705e-01 1.076295e-01
#> 87  2.563251e-24 9.991232e-01 8.767892e-04
#> 89  6.056125e-21 9.999467e-01 5.331421e-05
#> 96  1.287091e-20 9.999774e-01 2.263433e-05
#> 98  3.209382e-21 9.999763e-01 2.372830e-05
#> 99  1.435623e-11 1.000000e+00 2.855331e-09
#> 102 4.065940e-44 3.298961e-04 9.996701e-01
#> 105 4.119364e-53 4.170138e-07 9.999996e-01
#> 115 1.741312e-51 5.128539e-07 9.999995e-01
#> 119 8.184961e-68 2.919340e-10 1.000000e+00
#> 122 5.000649e-43 2.970267e-04 9.997030e-01
#> 123 6.677163e-58 2.075292e-07 9.999998e-01
#> 130 2.325154e-38 4.624754e-02 9.537525e-01
#> 132 4.215852e-43 1.614612e-04 9.998385e-01
#> 133 3.249585e-52 1.095243e-06 9.999989e-01
#> 143 4.065940e-44 3.298961e-04 9.996701e-01
#> 144 1.332725e-52 3.137510e-07 9.999997e-01
#> 147 1.176505e-40 6.848661e-03 9.931513e-01

Prediction as classification:

prediction <- predict(model, data.test , type = "class")
prediction
#>  [1] setosa     setosa     setosa     setosa     setosa     setosa    
#>  [7] setosa     setosa     setosa     setosa     versicolor versicolor
#> [13] versicolor versicolor versicolor versicolor versicolor versicolor
#> [19] versicolor virginica  versicolor versicolor versicolor versicolor
#> [25] versicolor versicolor virginica  virginica  virginica  virginica 
#> [31] virginica  virginica  virginica  virginica  virginica  virginica 
#> [37] virginica  virginica 
#> Levels: setosa versicolor virginica

Confusion Matrix:

mc <- confusion.matrix(data.test, prediction)
mc
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12
#> 
#> Overall Accuracy: 0.9737
#> Overall Error:    0.0263
#> 
#> Category Accuracy:
#> 
#>        setosa   versicolor    virginica
#>      1.000000     0.937500     1.000000

Quadratic Discriminant Analysis

Modeling:

model <- train.qda(Species~., data.train)
model
#> Call:
#> qda(Species ~ ., data = data.train)
#> 
#> Prior probabilities of groups:
#>     setosa versicolor  virginica 
#>  0.3571429  0.3035714  0.3392857 
#> 
#> Group means:
#>            Sepal.Length Sepal.Width Petal.Length Petal.Width
#> setosa         4.985000    3.405000     1.450000    0.247500
#> versicolor     5.970588    2.764706     4.285294    1.344118
#> virginica      6.576316    3.000000     5.507895    2.015789

Prediction as probability:

prediction <- predict(model, data.test , type = "prob")
prediction
#>            setosa   versicolor    virginica
#> 1    1.000000e+00 6.691672e-32 1.889822e-56
#> 13   1.000000e+00 4.502161e-25 1.643751e-45
#> 16   1.000000e+00 2.425866e-44 9.946646e-77
#> 25   1.000000e+00 1.846038e-22 7.222161e-41
#> 28   1.000000e+00 2.263734e-31 1.405378e-55
#> 37   1.000000e+00 1.066101e-34 1.642074e-61
#> 40   1.000000e+00 2.223760e-29 1.409863e-52
#> 41   1.000000e+00 2.574092e-30 7.078165e-55
#> 45   1.000000e+00 5.261433e-25 1.362316e-47
#> 48   1.000000e+00 5.965990e-25 9.832492e-46
#> 51   3.913327e-93 9.999983e-01 1.748560e-06
#> 54   1.132001e-65 9.957950e-01 4.205039e-03
#> 62   6.799811e-76 9.996645e-01 3.354597e-04
#> 63   1.586686e-60 9.999626e-01 3.741195e-05
#> 64   2.841930e-97 9.922291e-01 7.770865e-03
#> 67   3.382059e-92 9.774238e-01 2.257615e-02
#> 68   3.965797e-62 9.999593e-01 4.067484e-05
#> 80   2.706100e-39 1.000000e+00 2.529603e-08
#> 82   2.922149e-48 9.999955e-01 4.460501e-06
#> 84  1.063813e-125 3.546163e-02 9.645384e-01
#> 85   1.169232e-93 9.200247e-01 7.997532e-02
#> 87   1.016187e-96 9.999161e-01 8.390841e-05
#> 89   2.201163e-68 9.999721e-01 2.794462e-05
#> 96   6.738690e-70 9.999703e-01 2.968347e-05
#> 98   4.789342e-75 9.999871e-01 1.287034e-05
#> 99   4.964949e-27 9.999997e-01 3.192563e-07
#> 102 7.054324e-137 4.787216e-05 9.999521e-01
#> 105 6.250126e-188 1.197738e-07 9.999999e-01
#> 115 6.000194e-160 6.444145e-15 1.000000e+00
#> 119 1.837646e-268 2.990105e-10 1.000000e+00
#> 122 2.283010e-131 1.810862e-06 9.999982e-01
#> 123 2.854321e-243 1.329319e-07 9.999999e-01
#> 130 9.978738e-165 1.315865e-02 9.868413e-01
#> 132 2.577367e-212 3.871752e-02 9.612825e-01
#> 133 3.618142e-175 1.809676e-08 1.000000e+00
#> 143 7.054324e-137 4.787216e-05 9.999521e-01
#> 144 7.744264e-197 5.679973e-08 9.999999e-01
#> 147 1.109338e-127 7.516888e-05 9.999248e-01

Prediction as classification:

prediction <- predict(model, data.test , type = "class")
prediction
#>  [1] setosa     setosa     setosa     setosa     setosa     setosa    
#>  [7] setosa     setosa     setosa     setosa     versicolor versicolor
#> [13] versicolor versicolor versicolor versicolor versicolor versicolor
#> [19] versicolor virginica  versicolor versicolor versicolor versicolor
#> [25] versicolor versicolor virginica  virginica  virginica  virginica 
#> [31] virginica  virginica  virginica  virginica  virginica  virginica 
#> [37] virginica  virginica 
#> Levels: setosa versicolor virginica

Confusion Matrix:

mc <- confusion.matrix(data.test, prediction)
mc
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12
#> 
#> Overall Accuracy: 0.9737
#> Overall Error:    0.0263
#> 
#> Category Accuracy:
#> 
#>        setosa   versicolor    virginica
#>      1.000000     0.937500     1.000000

Random Forest

Modeling:

model <- train.randomForest(Species~., data.train)
model
#> 
#> Call:
#>  randomForest(formula = Species ~ ., data = data.train, importance = TRUE) 
#>                Type of random forest: classification
#>                      Number of trees: 500
#> No. of variables tried at each split: 2
#> 
#>         OOB estimate of  error rate: 5.36%
#> Confusion matrix:
#>            setosa versicolor virginica class.error
#> setosa         40          0         0  0.00000000
#> versicolor      0         31         3  0.08823529
#> virginica       0          3        35  0.07894737

Prediction as probability:

prediction <- predict(model, data.test , type = "prob")
prediction
#>     setosa versicolor virginica
#> 1    1.000      0.000     0.000
#> 13   1.000      0.000     0.000
#> 16   0.990      0.010     0.000
#> 25   1.000      0.000     0.000
#> 28   1.000      0.000     0.000
#> 37   0.960      0.040     0.000
#> 40   1.000      0.000     0.000
#> 41   1.000      0.000     0.000
#> 45   1.000      0.000     0.000
#> 48   1.000      0.000     0.000
#> 51   0.000      0.982     0.018
#> 54   0.000      0.986     0.014
#> 62   0.002      0.990     0.008
#> 63   0.000      0.942     0.058
#> 64   0.000      0.976     0.024
#> 67   0.010      0.972     0.018
#> 68   0.000      1.000     0.000
#> 80   0.000      0.998     0.002
#> 82   0.000      1.000     0.000
#> 84   0.000      0.190     0.810
#> 85   0.042      0.896     0.062
#> 87   0.000      0.988     0.012
#> 89   0.010      0.988     0.002
#> 96   0.008      0.990     0.002
#> 98   0.000      0.994     0.006
#> 99   0.002      0.966     0.032
#> 102  0.000      0.066     0.934
#> 105  0.000      0.000     1.000
#> 115  0.000      0.054     0.946
#> 119  0.000      0.006     0.994
#> 122  0.002      0.192     0.806
#> 123  0.000      0.000     1.000
#> 130  0.000      0.322     0.678
#> 132  0.000      0.000     1.000
#> 133  0.000      0.000     1.000
#> 143  0.000      0.066     0.934
#> 144  0.000      0.000     1.000
#> 147  0.000      0.054     0.946

Prediction as classification:

prediction <- predict(model, data.test , type = "class")
prediction
#>  [1] setosa     setosa     setosa     setosa     setosa     setosa    
#>  [7] setosa     setosa     setosa     setosa     versicolor versicolor
#> [13] versicolor versicolor versicolor versicolor versicolor versicolor
#> [19] versicolor virginica  versicolor versicolor versicolor versicolor
#> [25] versicolor versicolor virginica  virginica  virginica  virginica 
#> [31] virginica  virginica  virginica  virginica  virginica  virginica 
#> [37] virginica  virginica 
#> Levels: setosa versicolor virginica

Confusion Matrix:

mc <- confusion.matrix(data.test, prediction)
mc
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12
#> 
#> Overall Accuracy: 0.9737
#> Overall Error:    0.0263
#> 
#> Category Accuracy:
#> 
#>        setosa   versicolor    virginica
#>      1.000000     0.937500     1.000000

The model still supports the functions of the original package.

library(randomForest)
varImpPlot(model)

K-Nearest Neighbors

Modeling:

model <- train.knn(Species~., data.train)
model
#> 
#> Call:
#> kknn::train.kknn(formula = Species ~ ., data = data.train)
#> 
#> Type of response variable: nominal
#> Minimal misclassification: 0.05357143
#> Best kernel: optimal
#> Best k: 6

Prediction as probability:

prediction <- predict(model, data.test , type = "prob")
prediction
#>       setosa versicolor  virginica
#>  [1,]      1 0.00000000 0.00000000
#>  [2,]      1 0.00000000 0.00000000
#>  [3,]      1 0.00000000 0.00000000
#>  [4,]      1 0.00000000 0.00000000
#>  [5,]      1 0.00000000 0.00000000
#>  [6,]      1 0.00000000 0.00000000
#>  [7,]      1 0.00000000 0.00000000
#>  [8,]      1 0.00000000 0.00000000
#>  [9,]      1 0.00000000 0.00000000
#> [10,]      1 0.00000000 0.00000000
#> [11,]      0 1.00000000 0.00000000
#> [12,]      0 1.00000000 0.00000000
#> [13,]      0 0.88155533 0.11844467
#> [14,]      0 0.88155533 0.11844467
#> [15,]      0 0.88155533 0.11844467
#> [16,]      0 1.00000000 0.00000000
#> [17,]      0 1.00000000 0.00000000
#> [18,]      0 1.00000000 0.00000000
#> [19,]      0 1.00000000 0.00000000
#> [20,]      0 0.08866211 0.91133789
#> [21,]      0 0.97854845 0.02145155
#> [22,]      0 1.00000000 0.00000000
#> [23,]      0 1.00000000 0.00000000
#> [24,]      0 1.00000000 0.00000000
#> [25,]      0 1.00000000 0.00000000
#> [26,]      0 1.00000000 0.00000000
#> [27,]      0 0.00000000 1.00000000
#> [28,]      0 0.00000000 1.00000000
#> [29,]      0 0.00000000 1.00000000
#> [30,]      0 0.00000000 1.00000000
#> [31,]      0 0.02145155 0.97854845
#> [32,]      0 0.00000000 1.00000000
#> [33,]      0 0.08866211 0.91133789
#> [34,]      0 0.00000000 1.00000000
#> [35,]      0 0.00000000 1.00000000
#> [36,]      0 0.00000000 1.00000000
#> [37,]      0 0.00000000 1.00000000
#> [38,]      0 0.17779340 0.82220660

Prediction as classification:

prediction <- predict(model, data.test , type = "class")
prediction
#>  [1] setosa     setosa     setosa     setosa     setosa     setosa    
#>  [7] setosa     setosa     setosa     setosa     versicolor versicolor
#> [13] versicolor versicolor versicolor versicolor versicolor versicolor
#> [19] versicolor virginica  versicolor versicolor versicolor versicolor
#> [25] versicolor versicolor virginica  virginica  virginica  virginica 
#> [31] virginica  virginica  virginica  virginica  virginica  virginica 
#> [37] virginica  virginica 
#> Levels: setosa versicolor virginica

Confusion Matrix:

mc <- confusion.matrix(data.test, prediction)
mc
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12
#> 
#> Overall Accuracy: 0.9737
#> Overall Error:    0.0263
#> 
#> Category Accuracy:
#> 
#>        setosa   versicolor    virginica
#>      1.000000     0.937500     1.000000

Neural Networks (nnet)

Modeling:

model <- train.nnet(Species~., data.train, size = 20)
#> # weights:  163
#> initial  value 129.434889 
#> iter  10 value 11.287791
#> iter  20 value 0.777812
#> iter  30 value 0.003981
#> final  value 0.000060 
#> converged
model
#> a 4-20-3 network with 163 weights
#> inputs: Sepal.Length Sepal.Width Petal.Length Petal.Width 
#> output(s): Species 
#> options were - softmax modelling

Prediction as probability:

prediction <- predict(model, data.test , type = "prob")
prediction
#>           setosa   versicolor    virginica
#> 1   9.999986e-01 1.383479e-06 3.359731e-19
#> 13  9.999986e-01 1.383636e-06 3.360458e-19
#> 16  9.999986e-01 1.383395e-06 3.359270e-19
#> 25  9.999977e-01 2.308408e-06 5.338393e-19
#> 28  9.999986e-01 1.383463e-06 3.359620e-19
#> 37  9.999986e-01 1.383435e-06 3.359494e-19
#> 40  9.999986e-01 1.383488e-06 3.359739e-19
#> 41  9.999986e-01 1.383499e-06 3.359850e-19
#> 45  9.999986e-01 1.400473e-06 3.396849e-19
#> 48  9.999986e-01 1.383654e-06 3.360625e-19
#> 51  1.115595e-17 1.000000e+00 3.859792e-16
#> 54  6.704809e-19 1.000000e+00 3.966570e-17
#> 62  7.120670e-18 1.000000e+00 2.691104e-16
#> 63  1.926870e-18 1.000000e+00 9.406836e-17
#> 64  8.197192e-19 1.000000e+00 4.702770e-17
#> 67  9.644722e-19 1.000000e+00 5.356778e-17
#> 68  7.710825e-18 1.000000e+00 2.870531e-16
#> 80  4.798027e-17 1.000000e+00 1.245300e-15
#> 82  9.190669e-18 1.000000e+00 3.305507e-16
#> 84  6.998721e-29 9.783636e-28 1.000000e+00
#> 85  7.113058e-19 1.000000e+00 4.661924e-17
#> 87  3.406634e-18 1.000000e+00 1.488599e-16
#> 89  1.106838e-17 1.000000e+00 3.837079e-16
#> 96  9.812261e-18 1.000000e+00 3.483441e-16
#> 98  7.046697e-18 1.000000e+00 2.669149e-16
#> 99  6.250023e-17 1.000000e+00 1.540263e-15
#> 102 5.051812e-29 6.016841e-27 1.000000e+00
#> 105 4.590677e-29 1.074873e-26 1.000000e+00
#> 115 4.563998e-29 1.059431e-26 1.000000e+00
#> 119 4.294868e-29 1.509395e-26 1.000000e+00
#> 122 5.654063e-29 3.201470e-27 1.000000e+00
#> 123 4.399819e-29 1.411425e-26 1.000000e+00
#> 130 1.137869e-28 1.792370e-28 1.000000e+00
#> 132 1.302765e-28 3.242616e-29 1.000000e+00
#> 133 4.528324e-29 1.149565e-26 1.000000e+00
#> 143 5.051812e-29 6.016841e-27 1.000000e+00
#> 144 5.035766e-29 6.572968e-27 1.000000e+00
#> 147 6.053025e-29 2.281264e-27 1.000000e+00

Prediction as classification:

prediction <- predict(model, data.test , type = "class")
prediction
#>  [1] setosa     setosa     setosa     setosa     setosa     setosa    
#>  [7] setosa     setosa     setosa     setosa     versicolor versicolor
#> [13] versicolor versicolor versicolor versicolor versicolor versicolor
#> [19] versicolor virginica  versicolor versicolor versicolor versicolor
#> [25] versicolor versicolor virginica  virginica  virginica  virginica 
#> [31] virginica  virginica  virginica  virginica  virginica  virginica 
#> [37] virginica  virginica 
#> Levels: setosa versicolor virginica

Confusion Matrix:

mc <- confusion.matrix(data.test, prediction)
mc
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12
#> 
#> Overall Accuracy: 0.9737
#> Overall Error:    0.0263
#> 
#> Category Accuracy:
#> 
#>        setosa   versicolor    virginica
#>      1.000000     0.937500     1.000000

Neural Networks (neuralnet)

Modeling:

model <- train.neuralnet(Species~., data.train, hidden = c(5, 7, 6),
                         linear.output = FALSE, threshold = 0.01, stepmax = 1e+06)
summary(model)
#>                     Length Class      Mode    
#> call                  7    -none-     call    
#> response            336    -none-     logical 
#> covariate           448    -none-     numeric 
#> model.list            2    -none-     list    
#> err.fct               1    -none-     function
#> act.fct               1    -none-     function
#> linear.output         1    -none-     logical 
#> data                  5    data.frame list    
#> exclude               0    -none-     NULL    
#> net.result            1    -none-     list    
#> weights               1    -none-     list    
#> generalized.weights   1    -none-     list    
#> startweights          1    -none-     list    
#> result.matrix       139    -none-     numeric 
#> prmdt                 4    -none-     list

Prediction as probability:

prediction <- predict(model, data.test , type = "prob")
prediction
#>           setosa   versicolor    virginica
#> 1   1.000000e+00 3.446809e-09 9.207977e-31
#> 13  1.000000e+00 3.431848e-09 9.253007e-31
#> 16  1.000000e+00 3.464710e-09 9.154415e-31
#> 25  1.000000e+00 3.460573e-09 9.166492e-31
#> 28  1.000000e+00 3.439434e-09 9.230287e-31
#> 37  1.000000e+00 3.405075e-09 9.335313e-31
#> 40  1.000000e+00 3.439770e-09 9.229221e-31
#> 41  1.000000e+00 3.454044e-09 9.186200e-31
#> 45  1.000000e+00 3.464711e-09 9.154281e-31
#> 48  1.000000e+00 3.462756e-09 9.159953e-31
#> 51  9.056655e-21 1.000000e+00 5.242281e-13
#> 54  9.199862e-22 1.000000e+00 9.674819e-12
#> 62  1.843951e-21 1.000000e+00 2.715652e-12
#> 63  1.113558e-20 1.000000e+00 4.667777e-13
#> 64  1.017739e-23 1.000000e+00 1.054842e-09
#> 67  3.080147e-24 1.000000e+00 2.280508e-09
#> 68  1.194968e-20 1.000000e+00 3.954779e-13
#> 80  1.529933e-06 9.998006e-01 3.733476e-20
#> 82  1.367241e-20 1.000000e+00 3.509938e-13
#> 84  7.325186e-40 1.490605e-09 1.000000e+00
#> 85  1.919662e-24 1.000000e+00 3.244973e-09
#> 87  1.805485e-21 1.000000e+00 3.094284e-12
#> 89  5.810945e-21 1.000000e+00 7.608977e-13
#> 96  7.313367e-21 1.000000e+00 6.093775e-13
#> 98  6.938315e-21 1.000000e+00 7.077371e-13
#> 99  1.000000e+00 5.010612e-09 1.773402e-30
#> 102 4.169881e-43 4.300800e-13 1.000000e+00
#> 105 2.947217e-46 1.755565e-16 1.000000e+00
#> 115 1.171536e-42 7.565021e-13 1.000000e+00
#> 119 9.256015e-48 3.400952e-18 1.000000e+00
#> 122 1.832470e-40 3.287895e-10 1.000000e+00
#> 123 2.886545e-47 1.290550e-17 1.000000e+00
#> 130 1.450795e-37 4.000902e-07 9.999999e-01
#> 132 1.222779e-43 3.686602e-13 1.000000e+00
#> 133 5.054144e-46 2.750890e-16 1.000000e+00
#> 143 4.169881e-43 4.300800e-13 1.000000e+00
#> 144 2.261717e-46 1.431147e-16 1.000000e+00
#> 147 3.185724e-41 2.768763e-11 1.000000e+00

Prediction as classification:

prediction <- predict(model, data.test , type = "class")
prediction
#>  [1] setosa     setosa     setosa     setosa     setosa     setosa    
#>  [7] setosa     setosa     setosa     setosa     versicolor versicolor
#> [13] versicolor versicolor versicolor versicolor versicolor versicolor
#> [19] versicolor virginica  versicolor versicolor versicolor versicolor
#> [25] versicolor setosa     virginica  virginica  virginica  virginica 
#> [31] virginica  virginica  virginica  virginica  virginica  virginica 
#> [37] virginica  virginica 
#> Levels: setosa versicolor virginica

Confusion Matrix:

mc <- confusion.matrix(data.test, prediction)
mc
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      1         14         1
#>   virginica       0          0        12

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      1         14         1
#>   virginica       0          0        12
#> 
#> Overall Accuracy: 0.9474
#> Overall Error:    0.0526
#> 
#> Category Accuracy:
#> 
#>        setosa   versicolor    virginica
#>      1.000000     0.875000     1.000000

Support Vector Machines

Modeling:

model <- train.svm(Species~., data.train)
model
#> 
#> Call:
#> svm(formula = Species ~ ., data = data.train, probability = TRUE)
#> 
#> 
#> Parameters:
#>    SVM-Type:  C-classification 
#>  SVM-Kernel:  radial 
#>        cost:  1 
#> 
#> Number of Support Vectors:  47

Prediction as probability:

prediction <- predict(model, data.test , type = "prob")
prediction
#>          setosa  versicolor   virginica
#> 1   0.975860958 0.013757554 0.010381488
#> 13  0.968222765 0.019787248 0.011989988
#> 16  0.911034087 0.050381129 0.038584784
#> 25  0.969260887 0.018012711 0.012726402
#> 28  0.974556454 0.014977704 0.010465842
#> 37  0.968156507 0.019159888 0.012683605
#> 40  0.974464336 0.014874193 0.010661471
#> 41  0.975620430 0.013898904 0.010480665
#> 45  0.966907885 0.020991755 0.012100360
#> 48  0.970297256 0.016719339 0.012983404
#> 51  0.028534671 0.893620306 0.077845024
#> 54  0.009522061 0.943497629 0.046980311
#> 62  0.013106574 0.956087867 0.030805559
#> 63  0.020885185 0.966277260 0.012837554
#> 64  0.010694026 0.910217138 0.079088836
#> 67  0.014294620 0.903745978 0.081959402
#> 68  0.017701721 0.976563723 0.005734556
#> 80  0.024639959 0.968946465 0.006413576
#> 82  0.015993412 0.974233164 0.009773425
#> 84  0.009911426 0.239064489 0.751024085
#> 85  0.017928538 0.886676770 0.095394691
#> 87  0.017687662 0.871847924 0.110464414
#> 89  0.022392815 0.966109725 0.011497459
#> 96  0.022729064 0.968838507 0.008432430
#> 98  0.013399668 0.974466955 0.012133376
#> 99  0.048197572 0.931664546 0.020137883
#> 102 0.007959374 0.030697654 0.961342971
#> 105 0.009215605 0.002209793 0.988574602
#> 115 0.011886190 0.003735433 0.984378377
#> 119 0.056306085 0.034365448 0.909328467
#> 122 0.009643351 0.038670840 0.951685809
#> 123 0.031097542 0.026021173 0.942881286
#> 130 0.017786519 0.105533566 0.876679915
#> 132 0.032755368 0.044535218 0.922709414
#> 133 0.009636615 0.002294039 0.988069346
#> 143 0.007959374 0.030697654 0.961342971
#> 144 0.009302167 0.003272889 0.987424945
#> 147 0.010741481 0.060024226 0.929234293

Prediction as classification:

prediction <- predict(model, data.test , type = "class")
prediction
#>  [1] setosa     setosa     setosa     setosa     setosa     setosa    
#>  [7] setosa     setosa     setosa     setosa     versicolor versicolor
#> [13] versicolor versicolor versicolor versicolor versicolor versicolor
#> [19] versicolor virginica  versicolor versicolor versicolor versicolor
#> [25] versicolor versicolor virginica  virginica  virginica  virginica 
#> [31] virginica  virginica  virginica  virginica  virginica  virginica 
#> [37] virginica  virginica 
#> Levels: setosa versicolor virginica

Confusion Matrix:

mc <- confusion.matrix(data.test, prediction)
mc
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12
#> 
#> Overall Accuracy: 0.9737
#> Overall Error:    0.0263
#> 
#> Category Accuracy:
#> 
#>        setosa   versicolor    virginica
#>      1.000000     0.937500     1.000000

Extreme Gradient Boosting

Modeling:

model <- train.xgboost(Species~., data.train, nrounds = 79, maximize = FALSE, verbose = 0)
model
#> ##### xgb.Booster
#> raw: 139 Kb 
#> call:
#>   xgb.train(params = params, data = train_aux, nrounds = nrounds, 
#>     watchlist = watchlist, obj = obj, feval = feval, verbose = verbose, 
#>     print_every_n = print_every_n, early_stopping_rounds = early_stopping_rounds, 
#>     maximize = maximize, save_period = save_period, save_name = save_name, 
#>     xgb_model = xgb_model, callbacks = callbacks, eval_metric = "mlogloss")
#> params (as set within xgb.train):
#>   booster = "gbtree", objective = "multi:softprob", eta = "0.3", gamma = "0", max_depth = "6", min_child_weight = "1", subsample = "1", colsample_bytree = "1", num_class = "3", eval_metric = "mlogloss", validate_parameters = "TRUE"
#> xgb.attributes:
#>   niter
#> callbacks:
#>   cb.evaluation.log()
#> # of features: 4 
#> niter: 79
#> nfeatures : 4 
#> evaluation_log:
#>     iter train_mlogloss
#>        1       0.743578
#>        2       0.532534
#> ---                    
#>       78       0.015663
#>       79       0.015611

Prediction as probability:

prediction <- predict(model, data.test , type = "prob")
prediction
#>             setosa  versicolor    virginica
#>  [1,] 0.9944374561 0.004435789 0.0011267414
#>  [2,] 0.9939395189 0.004433568 0.0016269000
#>  [3,] 0.9821980596 0.016689112 0.0011128737
#>  [4,] 0.9944374561 0.004435789 0.0011267414
#>  [5,] 0.9944374561 0.004435789 0.0011267414
#>  [6,] 0.9837585092 0.015126886 0.0011146417
#>  [7,] 0.9944374561 0.004435789 0.0011267414
#>  [8,] 0.9944374561 0.004435789 0.0011267414
#>  [9,] 0.9944374561 0.004435789 0.0011267414
#> [10,] 0.9944374561 0.004435789 0.0011267414
#> [11,] 0.0009844096 0.997911751 0.0011038530
#> [12,] 0.0046227896 0.989484012 0.0058931801
#> [13,] 0.0017534802 0.997382224 0.0008642558
#> [14,] 0.0026980436 0.984013975 0.0132879959
#> [15,] 0.0013079111 0.996260285 0.0024318874
#> [16,] 0.0073815910 0.988237202 0.0043811873
#> [17,] 0.0028598958 0.994277835 0.0028622176
#> [18,] 0.0028598958 0.994277835 0.0028622176
#> [19,] 0.0046227896 0.989484012 0.0058931801
#> [20,] 0.0031536364 0.014520043 0.9823263288
#> [21,] 0.0272189658 0.956625819 0.0161552429
#> [22,] 0.0010063513 0.997825563 0.0011681097
#> [23,] 0.0054605692 0.992357016 0.0021824159
#> [24,] 0.0054605692 0.992357016 0.0021824159
#> [25,] 0.0011453496 0.997244596 0.0016100713
#> [26,] 0.0163861588 0.965533018 0.0180808567
#> [27,] 0.0038726968 0.004364253 0.9917631149
#> [28,] 0.0012481194 0.003202455 0.9955494404
#> [29,] 0.0038726968 0.004364253 0.9917631149
#> [30,] 0.0004997505 0.001309848 0.9981903434
#> [31,] 0.0065130251 0.026439615 0.9670473933
#> [32,] 0.0004997505 0.001309848 0.9981903434
#> [33,] 0.0077368501 0.124345370 0.8679177165
#> [34,] 0.0013536282 0.003473172 0.9951731563
#> [35,] 0.0004507682 0.001181465 0.9983677268
#> [36,] 0.0038726968 0.004364253 0.9917631149
#> [37,] 0.0013536282 0.003473172 0.9951731563
#> [38,] 0.0012919764 0.005974662 0.9927333593

Prediction as classification:

prediction <- predict(model, data.test , type = "class")
prediction
#>  [1] setosa     setosa     setosa     setosa     setosa     setosa    
#>  [7] setosa     setosa     setosa     setosa     versicolor versicolor
#> [13] versicolor versicolor versicolor versicolor versicolor versicolor
#> [19] versicolor virginica  versicolor versicolor versicolor versicolor
#> [25] versicolor versicolor virginica  virginica  virginica  virginica 
#> [31] virginica  virginica  virginica  virginica  virginica  virginica 
#> [37] virginica  virginica 
#> Levels: setosa versicolor virginica

Confusion Matrix:

mc <- confusion.matrix(data.test, prediction)
mc
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12

Some Rates:

general.indexes(mc = mc)
#> 
#> Confusion Matrix:
#>             prediction
#> real         setosa versicolor virginica
#>   setosa         10          0         0
#>   versicolor      0         15         1
#>   virginica       0          0        12
#> 
#> Overall Accuracy: 0.9737
#> Overall Error:    0.0263
#> 
#> Category Accuracy:
#> 
#>        setosa   versicolor    virginica
#>      1.000000     0.937500     1.000000