# How do the tolerances work in the Optimization Toolbox?

34 views (last 30 days)
MathWorks Support Team on 1 Oct 2012
Answered: Hesham on 28 Sep 2019
I would like to know how the tolerances work in the Optimization Toolbox.
For example, I would like to know if they they are absolute or relative. I would also like to know how they are checked.

MathWorks Support Team on 1 Oct 2012
This enhancement has been made for Release 14 (R14). For previous product releases, read below for any possible workarounds:
The tolerances in the Optimization Toolbox are:
TolX - tolerance on x
TolCon - tolerance on the Constraints
TolFun - tolerance on the objective function
TolPCG also exists, but this discussion focuses on the other three tolerances. All are mentioned at the following URL:
For most of the Optimization Toolbox functions, once one tolerance is met, the optimization stops (i.e. using an "OR" rule on tolerances) and provides you with a message indicating which tolerance was met. In the case where the function takes the option argument TolCon, this tolerance also must be met (i.e., the point must be feasible) for the algorithm to stop. This allows you to know which tolerance you must adjust in order to change the results you're seeing. The FMINSEARCH function, however, requires that both the TolX and TolFun tolerances be met before it stops.
The tolerances are stated in absolute terms, so as long as your tolerances are of a similar magnitude as the values involved (objective function and parameters), you're in good shape.
Use of the tolerances should be viewed as a way of controlling how much work the optimizer puts into solving a specific problem. On a relative level, the result obtained by a tighter (smaller) tolerance will involve more work on the part of the optimization algorithm than that required with a looser (larger) tolerance. However, the absolute relation between two tolerances and the two results they produce isn't straightforward. This is because the tolerance checks involve both the related parameters (for example, TolFun involves both the tolerance and the value of the objective function), as well as algorithm specific parameters calculated during the optimization. Therefore to completely understand the tolerance checks, some understanding of the algorithm involved is useful (especially for the large-scale, trust region based methods).
In practice, you can start with the default set of tolerances and examine their results. For example, to find a smaller minimum, you can lower TolFun; to get more precision on the parameters, lower TolX. In this way, the actual value of the tolerances isn't as important as the end result that you're getting. This allows you to focus on obtaining the level of accuracy required by the problem you're solving, which allows you to use up just as much computation time as needed.
The default values for the tolerances change for different functions. To determine what the default tolerance for a specific function, use the OPTIMSET function followed by the name of the specified function. For example,
optimset fmincon
optimset fzero
Because a good understanding of the algorithm can be of help in understanding the algorithm's results, we expect to document this in more detail in a later printing of the Optimization Toolbox User's Guide.

#### 1 Comment

jin wang on 5 Jan 2018
The answer is really helpful. Thank you. I have a doubt that really need your help to clarify.
You have mentioned the tolerance is stated in absolute form, lets say TolX = 10^-2. What if my parameters being optimized are of different scales, I don't think the absolute tolerance will be appropriate here, right?
But I don't know how to use relative tolerance like this: xi - xi+1 < TolX * (1 + f(xi)) how can I use the relative tolerance instead absolute one in, for example, fminsearch?

Hesham on 28 Sep 2019