Setup an optimization problem using Bayesian Optimization

Dear all,

Problem:I try to find the optimal set of variables using Bayesian optimization.

Before I go further into the depth, let me explain the background of what I am trying to achieve here.

I first did some physical experiments. These physical experiments give me certain results (i.e. objectives). For example I measured a total mass of 27 kg. I measured other results as well.

Then, in a simulation I try to recreate the results of the physical experiments. In those simulations I define the range and interval of the variables I try to optimize. The variables I try to optimize are mu_s (ranging from 0.1 to 0.5 with an interval of 0.1) and mu_r (ranging from 0.1 to 0.5 with an interval of 0.1).

The simulations will give me predicted results for certain combinations of variables. For example:

mu_s = 0.1 and mu_r = 0.7 will give me a predicted total mass of 29.45 kg.

Then I want to use Bayesian optimization to find the optimal set of variables that can predict the results of the physical experiment as close as possible using the data I have available from the simulations.

With the documentation from the links below I tried to make a Bayesian optimization, but I am kind of stuck on how to define the objective function.

https://nl.mathworks.com/help/deeplearning/ug/deep-learning-using-bayesian-optimization.html

https://nl.mathworks.com/help/stats/bayesianoptimization.html

https://itectec.com/matlab/matlab-is-there-a-worked-example-for-bayesian-optimization-with-a-classical-neural-network/

This is how far I got

optimVars = [
    optimizableVariable('\mu_s',[0.1 0.5])
    optimizableVariable('\mu_r',[0.1 0.5])];
objFcn = makeObjFcn(

Something to point out:

It is a single-objective problem. The objective function is defined as the total relative discrepancies between the simulation outputs and the experimental measurements using weighted sum. The weights of error components are equally contributing to the total simulation error. Thus, the weights of each term are set to be the same.

To search for new points to be evaluated next by the objective function, I want to use a combined acquisition function. First, we look for a point that maximizes Expected Improvement (EI), then for a point that maximizes Probability of Improvement. Thus, two new points are determined per iteration, which can then be evaluated in parallel. I think by using 'UseParallel' this can be achieved.

I looked at the global optimization tool under de section surrogate modelling, but I think that is not quite what I want.

Answer 1

[Matt J]

The functions you have been looking at (e.g. bayesopt) don't appear to be general model fitting solvers. They look like they are designed very specifically for training machine learning systems.

I think you might be able to achieve your ultimate goals by using either lsqnonlin,

https://www.mathworks.com/help/optim/ug/lsqnonlin.html

or lsqcurvefit (which does a lot more than just curve fitting),

https://www.mathworks.com/help/optim/ug/lsqcurvefit.html?s_tid=srchtitle

even if they do not implement precisely the search algorithm you have described.

One thing I would note - you will find that Matlab documentation only applies the terminology "multi-objective" to problems where you are trying to solve not just for a single solution, but a Pareto front. That doesn't seem to be the case for you, but if it is, have a look here:

https://www.mathworks.com/help/gads/multiobjective-optimization.html?s_tid=CRUX_lftnav

[Tessa Kol]

I am trying to achieve the same results as in the attached document.

I am doubting now wheter it is a multi-objective problem or a single-objective problem. I am trying to minimize the simulation error.

[Matt J]

Well, I'm not at all familiar with that paper, but the final paragraph of Section 4.2 does say something about Pareto-efficiency, so maybe you should look at paretosearch().

[Tessa Kol]

After further research I discovered that they used the weighted sum model to combine the different error components into a single objective function.

Single relative error component:

Objective:

So now that I defined my objective. How can I use Bayesian optimization? Bayesian optimization is surrogate modelling combined with gaussian process regressian to find the global optimimum (i.e. the optimal combination set of variables).

I don't come across examples or documentation where they use data sets to achieve optimization.

[Matt J]

This FEX contribution looks relevant and popular. I haven't used it myself, though.

https://www.mathworks.com/matlabcentral/fileexchange/66588-multi-objective-optimization-algorithm-for-expensive-to-evaluate-function

[Matt J]

The GitHub version seems to contain better usage instructions

https://github.com/Eric-Bradford/TS-EMO