GPopt (R)

This is the R version of Python's Techtonique/GPopt (https://docs.techtonique.net/GPopt/GPopt.html), a package for 'Bayesian' optimization of black-box functions (and machine learning hyperparameter tuning) using Gaussian Process Regression and other surrogate models.

Keep in mind that this package is for Machine Learning hyperparameter tuning. The global minimum won't always be found, but it isn't an issue, since it means you aren't overfitting the training set.

It's ported the same way as nnetsauce for R was: with uv to create an isolated Python virtual environment containing the Python GPopt package, and reticulate to call into it from R. Every function in this R package is a thin wrapper that returns the underlying Python object; the general rule is: object accesses with .'s in Python are replaced by $'s in R.

See this post for the technique: Finally figured out a way to port python packages to R using uv and reticulate.

Install

1. Create the Python virtual environment with uv

# pip install uv # if necessary
uv venv venv
source venv/bin/activate      # on Windows: venv\Scripts\activate
uv pip install pip GPopt

Keep track of where venv/ lives -- you'll pass its path as venv_path to every function in this package.

2. Install the R package

install.packages("remotes")
remotes::install_github("Techtonique/GPopt_r") # or wherever this package is hosted

reticulate will be installed automatically as a dependency.

Examples

Minimizing the Branin function

library(GPopt)

branin <- function(x) {
  x1 <- x[1]; x2 <- x[2]
  term1 <- (x2 - (5.1 * x1^2) / (4 * pi^2) + (5 * x1) / pi - 6)^2
  term2 <- 10 * (1 - 1 / (8 * pi)) * cos(x1)
  term1 + term2 + 10
}

opt <- GPOpt(
  lower_bound = c(-5, 0),
  upper_bound = c(10, 15),
  objective_func = branin,
  n_init = 10,
  n_iter = 40,
  venv_path = "./venv"
)

opt$optimize(verbose = 1L)
print(opt$x_min)  # current best parameters
print(opt$y_min)  # current best objective value

Tuning a scikit-learn model's hyperparameters

library(GPopt)

sklearn <- get_sklearn(venv_path = "./venv")
RandomForestClassifier <- sklearn$ensemble$RandomForestClassifier

X <- as.matrix(iris[, 1:4])
y <- as.integer(iris$Species) - 1L

mlopt <- MLOptimizer(scoring = "accuracy", cv = 5, venv_path = "./venv")

param_config <- list(
  n_estimators = list(bounds = c(10, 300), dtype = "int"),
  max_depth    = list(bounds = c(1, 20),   dtype = "int")
)

mlopt$optimize(
  X_train = X, y_train = y,
  estimator_class = RandomForestClassifier(),
  param_config = param_config,
  verbose = 1L
)

print(mlopt$get_best_parameters())
print(mlopt$get_best_score())

Bayesian optimization with early stopping

library(GPopt)

opt <- BOstopping(
  f = branin,
  bounds = rbind(c(-5, 10), c(0, 15)),
  venv_path = "./venv"
)
result <- opt$optimize(n_iter = 100L)

Using a custom (conformalized) surrogate model

library(GPopt)

sklearn <- get_sklearn(venv_path = "./venv")
ns <- get_nnetsauce(venv_path = "./venv")

opt <- GPOpt(
  lower_bound = c(-5, 0),
  upper_bound = c(10, 15),
  objective_func = branin,
  acquisition="ucb",
  method="splitconformal",
  surrogate_obj = ns$PredictionInterval(sklearn$ensemble$RandomForestRegressor()),
  venv_path = "./venv"
)
opt$optimize(verbose = 1L)

Functions in this package

• GPOpt() -- main Bayesian optimizer (wraps GPopt.GPOpt)
• MLOptimizer() -- cross-validated hyperparameter tuning for scikit-learn estimators (wraps GPopt.MLOptimizer)
• GenericSurrogate() -- wrap any scikit-learn-compatible regressor as a surrogate (wraps GPopt.GenericSurrogate)
• GeneralizationOpt() -- diagnostics for the generalization gap of hyperparameters (wraps GPopt.GeneralizationOpt)
• BOstopping() -- Bayesian optimization with early stopping (wraps GPopt.BOstopping)
• get_GPopt(), get_numpy(), get_sklearn() -- access the raw Python modules for anything not (yet) covered above

Notes

• Every function takes a venv_path argument pointing at the uv-created virtual environment; this must be the same environment across a given script/session for object identity between calls to work correctly.
• R functions passed as objective_func (or f) are called from Python with a 1-D numpy array converted back to an R numeric vector -- so index with x[1], x[2], ... as usual in R, not x[0], x[1], ...
• Since this package hands you the underlying Python object, the full Python GPopt API is available -- consult the Python package's documentation and blog posts for anything not covered in this README.

License

BSD-3-Clause-Clear, matching the original Python GPopt package and nnetsauce_r.