bayesopt with equality and inquality constraints

How to optimize the following objective funciton with constraints?

min f(x)

s.t. Ax<b

x(1)=3;

x(1)<x(2)

Here are my codes:

global A b

A=eye(2);

b=[3, 3]

xdata=[

optimizableVariable('x1', [-3 ; 6], 'type', 'real')

optimizableVariable('x2', [-1; 6], 'type', 'real')];

results=bayesopt(@fun, xdata, 'IsObjectiveDeterministic',true, 'XConstraintFcn',@xconstraint);

function objective=fun(x)

global A

objective=[x.x1, x.x2]*A*[x.x1; x.x2]+3

end

function tf=xconstraint(x)

global A b

tf1=x.x1==3

tf2=x.x1<x.x2;

tf3=A*[x.x1; x.x2]<b

tf=tf1&tf2&tf3;

end

However, matlab reports error:

Error using *

Incorrect dimensions for matrix multiplication. Check that the number of columns in the first matrix matches the number of

rows in the second matrix. To perform elementwise multiplication, use '.*'.

I was wondering how to implement the inquality constraints (in matrix form like quadprog) in bayesian optimization?

Besides, are there any ways to remove the global variables?

Thanks!

Answer 1

[Alan Weiss]

I believe that you have two errors in your formulation. As explained in Constraints in Bayesian Optimization, bayesopt attempts to pass thousands of points to your constraint function to obtain the feasibility results as logical values. So you should write your XConstraintFcn to take an arbitrary number of input variables. You can write a loop to calculate the values:

function tf = xconstraint(x,A,b)
global A b
n = size(x,1);
tf = false(n,1);
for ii = 1:n
    tf(i,1) = x.x1(i) < x.x2(i);
    tf(i,1) = all(A*[x.x1(i); x.x2(i)] < b) & tf(i,1);
end
end

You will notice that I have three arguments for xconstraint(x,A,b). This is to get around the problem of global variables. You call the function as @(x)xconstraint(x,A,b). For details, see Parameterizing Functions.

The other error is trying to set x1 = 3 while having it declared as a real optimizable variable. This is impossible. You need to set it as either a constant, so there is no x1 as an optimizable variable, or you need to set the lower and upper bounds equal to 3, or you need to set it as a categorical variable with value 3. If it is a real variable with unequal bounds, bayesopt will try to set it to random real values, and the Xconstraint function will return that the value is not acceptable, and your optimization will never get anywhere.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

[Liao Peizhi]

Hi Alan,

Thank you for your patience and answers!

I also notice that the size of x in constraint function is 10000. I was wondering what does this number stand for? Is it related to the 'NumSeedPoints' ? However, the default value for seed points is 4.

Besides, will it (10000) slow down the calculation and is it possible to lower this number?

Thank you again for your kind help!

Best regards,

Peizhi Liao

[Alan Weiss]

I tried to be as clear as I could when describing deterministic constraints in the doocumentation. bayesopt samples these constraints at a large number of points in order to find points in the feasible region. These points are pseudorandom, and this is why you cannot set x1 = 3 in this way, because that test would always return false. These points have nothing to do with seed points. The evalutaion time of these points is very low, in general, especially compared to everything else going on in the solver.

I also just noticed that I left a global statement in my answer. That should not be there. In general, you should not use global variables.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

[Federico]

I have a problem related to this.

I have a certain number of decisional variables (N) in the Bayesian Optimization.

I need the sum of these variables to be smaller than a certain threshold (B), yet each variable has to be generated within the domain [0,B]. Now, I tried to program this through deterministic constraints.

Now, I am able to find some variables satisfying the contraint if I choose N=3, but ideally I would need this number to be more than 10, and this makes impossible to generate set of parameters satisfying the contraint, since the algorithm "only" generates 10000 combinations.

Is there a way to modify this number so to satisfy the constraint with a large number of decisional variables?

Thank you in advance!

[Alan Weiss]

i suggest that you generate random samples in the domain . The reason is this. For large N, a set of samples that satisfies and will generally have . Therefore it is safe to sample from the much-reduced sample space. In fact, depending on N, you can choose any number larger than 1 as the multiplier; I chose 2 because it is relatively small and should be sufficient for your purpose.

For the mathematical reason this is true, see any book on large deviations.

Alan Weiss

MATLAB mathematical toolbox documentation

[Federico]

Thank you for your answer.

Unfortunately, this is not really viable to me. I really need to test any possible set of parameters, for instance, let's set N=3, and B=1. I would need to have as a possible set of parameters the following (as they could be an optimal solution), for instance.

N1= 1-eps

N2 = eps/2

N3 = eps/2

With eps really small (let's say eps=1/10000). If I would choose to generate the variables in the set [0,2/3] such set of N1,N2,N3 would not be evaluated.

[Alan Weiss]

I'm sorry, but I really don't understand your issues. I suggest that you start a new question and put in the question exactly what you are trying to do.

Alan Weiss

MATLAB mathematical toolbox documentation

[Federico]

All right! I created an ad-hoc topic for my question, hope this helps.

Thank you again.