# Alternating optimization

The {ao} package implements a numerical optimization algorithm called alternating optimization in R.

Alternating optimization is an iterative procedure which optimizes a function jointly over all parameters by alternately performing restricted optimization over individual parameter subsets.

For additional details on the method, please refer to the package vignette.

## Installation

You can install the released version of {ao} from CRAN with:

install.packages("ao")

## Example

The following is a simple example to perform alternating optimization of the Himmelblauâ€™s function, separately for $$x_1$$ and $$x_2$$, with the parameter restrictions $$-5 \leq x_1, x_2 \leq 5$$.

### Step 1: Load the package

library("ao")
#> Thanks for using {ao} 0.3.2, happy alternating optimization!
#> Documentation: https://loelschlaeger.de/ao

### Step 2: Define the function to be optimized

himmelblau <- function(x, a, b) (x[1]^2 + x[2] + a)^2 + (x[1] + x[2]^2 + b)^2

The function is optimized over its first argument (x), which needs to be a numeric vector. Other function arguments (a and b in this case) remain fixed during the optimization. The function should return a single numeric value.

### Step 3: Define a base optimizer

Alternating optimization requires a base optimizer that numerically solves the optimization problems in the partitions of the parameter vector. Such an optimizer must be defined through the framework provided by the {optimizeR} package, please see its documentation for details.

base_optimizer <- optimizeR::Optimizer$new(which = "stats::optim", lower = -5, upper = 5, method = "L-BFGS-B") ### Step 4: Call the ao() function Despite f and base_optimizer, which have been defined above, the ao() function requires the following arguments: • p defines the starting parameter values, • a and b are fixed function arguments, • partition defines the parameter subsets (here, the first entry of x and the second are optimized separately). ao(f = himmelblau, p = c(0, 0), a = -11, b = -7, partition = list(1, 2), base_optimizer = base_optimizer) #>$value
#> [1] 1.940035e-12
#>
#> $estimate #> [1] 3.584428 -1.848126 #> #>$sequence
#>    iteration partition        value      seconds       p1        p2
#> 1          0        NA 1.700000e+02 0.0000000000 0.000000  0.000000
#> 2          1         1 1.327270e+01 0.0094270706 3.395691  0.000000
#> 3          1         2 1.743666e+00 0.0010139942 3.395691 -1.803183
#> 4          2         1 2.847292e-02 0.0008139610 3.581412 -1.803183
#> 5          2         2 4.687472e-04 0.0006930828 3.581412 -1.847412
#> 6          3         1 7.368063e-06 0.0012190342 3.584381 -1.847412
#> 7          3         2 1.157612e-07 0.0004658699 3.584381 -1.848115
#> 8          4         1 1.900153e-09 0.0004830360 3.584427 -1.848115
#> 9          4         2 4.221429e-11 0.0003638268 3.584427 -1.848126
#> 10         5         1 3.598278e-12 0.0003728867 3.584428 -1.848126
#> 11         5         2 1.940035e-12 0.0003712177 3.584428 -1.848126
#>
#> \$seconds
#> [1] 0.01522398

The output contains:

• the function value at convergence,

• the parameter value estimate at convergence,

• sequence provides information about the updates in the single iterations and partitions,

• and the optimization time in seconds.

## Contact

Have a question, found a bug, request a feature, want to contribute? Please file an issue.