How are integer parameters dealt with in "bayesopt"?

How does the Bayesian Optimizer "bayesopt" deal with integer parameters and what optimizations are implemented for that purpose? The documentation page (https://www.mathworks.com/help/stats/bayesian-optimization-algorithm.html) only mentions the general bayesian optimization algorithm but does not mention anything regarding how parameters subjected to integer constraint are dealt with.

Answer 1

[MathWorks Support Team]

When fitting the internal surrogate models (i.e., the Gaussian Process models of the objective function and the constraints), bayesopt treats integer variables as though they are real-valued. It is as if each integer variable were a real-valued variable whose only observations happened to be integers.

When bayesopt maximizes the Acquisition function in order to choose the next point, it first chooses 10000 random real-valued points, and then each point has its integer variables rounded to the nearest integer. Then the acquisition function is evaluated on these 10000 points and the best 10 are chosen. Then for each of these 10 points, a Nelder-Mead simplex algorithm is run to conduct a local optimization. This is done using the fminsearch function (https://www.mathworks.com/help/matlab/math/optimizing-nonlinear-functions.html#bsgpq6p-11).

The objective function that bayesopt passes to fminsearch is basically the acquisition function with rounding built-in for the integer variables. That is, when fminsearch proposes new points to be evaluated, the objective function rounds the real values to integers before passing them to the acquisition function. Because Nelder-Mead is a gradient-free optimization algorithm, it has no trouble optimizing this noncontinuous objective function.