An Introduction to mvnfast

Introduction

The mvn R package provides computationally efficient tools related to the multivariate normal distribution. The tools are generally faster than those provided by other packages, thanks to the use of C++ code through the Rcpp\RcppArmadillo packages and parallelization through the OpenMP API. The most important functions are:

In the following sections we will benchmark each function against equivalent functions provided by other packages, while in the final section we provide an example application.

Simulating multivariate normal random vectors

Simulating multivariate normal random variables is an essential step in many Monte Carlo algorithms (such as MCMC or Particle Filters), hence this operations has to be as fast as possible. Here we compare the rmvn function with the equivalent function rmvnorm (from the mvtnorm package) and mvrnorm (from the MASS package). In particular, we simulate \(10^4\) twenty-dimensional random vectors:

library("microbenchmark")
library("mvtnorm")
library("mvnfast")
library("MASS")
# We might also need to turn off BLAS parallelism 
library("RhpcBLASctl")
blas_set_num_threads(1)

N <- 10000
d <- 20

# Creating mean and covariance matrix
mu <- 1:d
tmp <- matrix(rnorm(d^2), d, d)
mcov <- tcrossprod(tmp, tmp)

microbenchmark(rmvn(N, mu, mcov, ncores = 2),
               rmvn(N, mu, mcov),
               rmvnorm(N, mu, mcov),
               mvrnorm(N, mu, mcov))

## Unit: milliseconds
##                           expr       min        lq      mean    median
##  rmvn(N, mu, mcov, ncores = 2)  2.962333  3.052043  4.232991  3.158812
##              rmvn(N, mu, mcov)  5.054590  5.139264  7.313402  5.234082
##           rmvnorm(N, mu, mcov) 15.738991 16.465214 20.679759 16.851513
##           mvrnorm(N, mu, mcov) 15.357095 16.383270 21.587041 16.866174
##         uq      max neval cld
##   3.890893 30.91972   100 a  
##   5.819171 33.88493   100  b 
##  17.691762 45.88069   100   c
##  17.767322 45.23293   100   c

In this example rmvn cuts the computational time, relative to the alternatives, even when a single core is used. This gain is attributable to several factors: the use of C++ code and efficient numerical algorithms to simulate the random variables. Parallelizing the computation on two cores gives another appreciable speed-up. To be fair, it is necessary to point out that rmvnorm and mvrnorm have many more safety check on the user's input than rmvn. This is true also for the functions described in the next sections.

Finally, notice that this function does not use one of the Random Number Generators (RNGs) provided by R, but one of the parallel cryptographic RNGs described in (Salmon et al., 2011) and available here. It is important to point out that this RNG can safely be used in parallel, without risk of collisions between parallel sequence of random numbers, as detailed in the above reference.

Evaluating the multivariate normal density

Here we compare the dmvn function, which evaluates the multivariate normal density, with the equivalent function dmvtnorm (from the mvtnorm package). In particular we evaluate the density of \(10^4\) twenty-dimensional random vectors:

# Generating random vectors 
N <- 10000
d <- 20
mu <- 1:d
tmp <- matrix(rnorm(d^2), d, d)
mcov <- tcrossprod(tmp, tmp)
X <- rmvn(N, mu, mcov)

microbenchmark(dmvn(X, mu, mcov, ncores = 2),
               dmvn(X, mu, mcov),
               dmvnorm(X, mu, mcov))

## Unit: milliseconds
##                           expr      min       lq     mean   median
##  dmvn(X, mu, mcov, ncores = 2) 1.737373 1.833401 1.987410 1.886132
##              dmvn(X, mu, mcov) 2.870420 2.987661 3.166377 3.080808
##           dmvnorm(X, mu, mcov) 2.569005 2.710445 5.565932 3.204676
##        uq       max neval cld
##  2.011043  2.991912   100  a 
##  3.212795  3.875458   100  ab
##  3.856791 77.169832   100   b

Again, we get some speed-up using C++ code and some more from the parallelization.

Evaluating the Mahalanobis distance

Finally, we compare the maha function, which evaluates the square mahalanobis distance with the equivalent function mahalanobis (from the stats package). Also in the case we use \(10^4\) twenty-dimensional random vectors:

# Generating random vectors 
N <- 10000
d <- 20
mu <- 1:d
tmp <- matrix(rnorm(d^2), d, d)
mcov <- tcrossprod(tmp, tmp)
X <- rmvn(N, mu, mcov)

microbenchmark(maha(X, mu, mcov, ncores = 2),
               maha(X, mu, mcov),
               mahalanobis(X, mu, mcov))

## Unit: milliseconds
##                           expr      min       lq     mean   median
##  maha(X, mu, mcov, ncores = 2) 1.410446 1.496472 1.754015 1.636257
##              maha(X, mu, mcov) 2.553263 2.669778 2.908738 2.788488
##       mahalanobis(X, mu, mcov) 2.728863 2.886309 5.231537 3.773625
##        uq       max neval cld
##  1.880036  2.879330   100  a 
##  2.950655  5.755169   100  a 
##  4.153083 84.590688   100   b

The acceleration is similar to that obtained in the previous sections.

Example: mean-shift mode seeking algorithm

As an example application of the dmvn function, we implemented the mean-shift mode seeking algorithm. This procedure can be used to find the mode or maxima of a kernel density function, and it can be used to set up clustering algorithms. Here we simulate \(10^4\) d-dimensional random vectors from mixture of normal distributions: 

set.seed(5135)
N <- 10000
d <- 2
mu1 <- c(0, 0); mu2 <- c(2, 3)
Cov1 <- matrix(c(1, 0, 0, 2), 2, 2)
Cov2 <- matrix(c(1, -0.9, -0.9, 1), 2, 2)

bin <- rbinom(N, 1, 0.5)

X <- bin * rmvn(N, mu1, Cov1) + (!bin) * rmvn(N, mu2, Cov2)

Finally, we plot the resulting probability density and, starting from 10 initial points, we use mean-shift to converge to the nearest mode:

# Plotting
np <- 100
xvals <- seq(min(X[ , 1]), max(X[ , 1]), length.out = np)
yvals <- seq(min(X[ , 2]), max(X[ , 2]), length.out = np)
theGrid <- expand.grid(xvals, yvals) 
theGrid <- as.matrix(theGrid)
dens <- 0.5 * dmvn(theGrid, mu1, Cov1) + 0.5 * dmvn(theGrid, mu2, Cov2)
plot(X[ , 1], X[ , 2], pch = '.', lwd = 0.01, col = 3)
contour(x = xvals, y = yvals, z = matrix(dens, np, np),
        levels = c(0.002, 0.01, 0.02, 0.04, 0.08, 0.15 ), add = TRUE, lwd = 2)

# Mean-shift
library(plyr)
inits <- matrix(c(-2, 2, 0, 3, 4, 3, 2, 5, 2, -3, 2, 2, 0, 2, 3, 0, 0, -4, -2, 6), 
                10, 2, byrow = TRUE)
traj <- alply(inits,
              1,
              function(input)
                  ms(X = X, 
                     init = input, 
                     H = 0.05 * cov(X), 
                     ncores = 2, 
                     store = TRUE)$traj
              )

invisible( lapply(traj, 
                  function(input){ 
                    lines(input[ , 1], input[ , 2], col = 2, lwd = 1.5)
                    points(tail(input[ , 1]), tail(input[ , 2]))
           }))

plot of chunk mixPlot As we can see from the plot, each initial point leads one of two points that are very close to the true mode. Notice that the bandwidth for the kernel density estimator was chosen by trial-and-error, and less arbitrary choices are certainly possible in real applications.

References

h