|
|
|
| R2011b Documentation → Model-Based Calibration Toolbox | |
Learn more about Model-Based Calibration Toolbox |
|
|
|
|
| R2011b Documentation → Model-Based Calibration Toolbox | |
Learn more about Model-Based Calibration Toolbox |
|
| Contents | Index |
| On this page… |
|---|
Process for Analyzing Optimization Results |
This topic describes a process for analyzing the results from single-objective optimizations (e.g., maximizing torque vs. spark, ICP, ECP at an engine operating point, using foptcon, ga and patternsearch algorithms).
For each run of an optimization, the aim is to find the optimal solution. The Optimization Output View provides graphical tools to help you determine whether an optimal solution has been found for a given run. This view provides a table with icons that indicate the status of each optimization run.
Optimization Results Table Icons
| Icon | Description |
|---|---|
Green square Accept icon | Indicates success (algorithm exit flag > 0 ). |
Orange triangle Accept icon | Indicates the optimization terminated early (exit flag = 0 ). This situation typically occurs when the optimizer has reached some form of time limit. Examples of this include exceeding a number of iterations or function evaluation limit. In such cases, the optimization was in progress but was forced to terminate before the optimal solution had been found. |
Red circle Accept icon | Indicates failure (an exit flag < 0 ). Typically this occurs due to the problem being over constrained for this run. |
The process for analyzing point optimization results comprises the following tasks:
Switch to the Optimization Output view for the optimization.
Analyze all runs with green square Accept icons (
). For each run:
Inspect the Objective Graphs.
Inspect Objective Contour plots for as many pairs of free variables as possible. You can configure the optimization output view to display multiple contour plots simultaneously.
Has the solution found a local optimum? Many optimization algorithms are designed to locate local optima (e.g., foptcon in CAGE). Check each successful run to ensure that the optimizer has found the best solution possible. See Detecting Local Optima for more information and examples.
Does the optimization appear to have terminated early? In some cases an optimization appears to return sub-optimal results even though the optimizer has returned a positive exit flag. Investigate such cases. See Investigating Early Termination of Optimization.
Repeat steps 2a and 2b to analyze all runs with orange
triangle Accept icons (
) that indicate the optimization terminated
early. See Investigating Early Termination of Optimization for more
information and examples.
Repeat steps 2a and 2b to analyze all runs with red
circle Accept icons (
) that indicate failures. These runs
have typically failed to meet constraints. Inspect the plots and determine
if it is acceptable to relax any of the constraints.
After you investigate your results to identify problems, use these suggestions to try to improve your optimization results:
If you detect local optima, try running the optimization again to locate the best optimum.
Edit the initial condition manually for this optimization operating point and rerun.
For point optimizations that use the foptcon algorithm, set the Number of start points to be greater than 1 and rerun. In this case, CAGE performs the optimization more than once for each run. To save time, you might want to only repeat the offending runs in this way.
Use an alternative algorithm on the runs that have found a local optimum. For example, you could try the ga or patternsearch algorithms in CAGE (if you tried the foptcon algorithm first).
If the optimization terminates early:
In cases where the optimizer runs out of iterations/function evaluations/time and the solution returned is feasible, determine whether the solution is acceptable to you.
To accept the solution, select the Accept check box on the Optimization Results table.
If you reject the solution, rerun the optimization with modified parameter settings. In this case, if foptcon or patternsearch is being used, it is advisable to start the optimizer from the solution that has just been found.
In cases where the optimizer runs out of iterations/function evaluations/time and the solution returned is infeasible, you can try rerunning the optimization from different initial conditions (for foptcon or patternsearch ) or different parameter settings (all algorithms). If this approach does not resolve the problem, determine if any constraint has been violated. Investigate violated constraints, to determine whether they can be relaxed. If they can, rerun the optimization with the relaxed constraints; if not, leave the check box unselected to indicate the solution is unacceptable.
See also Handling Flat Optima.
The following figure shows views for an optimization which has found the optimal solution. The objective is to maximize Torque (Objective1) against spark angle (S), Exhaust valve closing (ECP) and Intake valve opening (ICP).
This result is taken from the Gasoline case study (see Gasoline Engine Calibration Case Study in the Model-Based Calibration Toolbox Getting Started Guide).
When you analyze the optimization results, look for results that have located the best optimum against the free variables.
In this case, an individual plot can only show it is highly likely rather than definitely the optimal value because there are more than two free variables. For problems with more than two free variables, the Objective Graphs and Contours views cannot guarantee that an optimal solution has been found because they provide projections of the model.
For further confirmation, you should inspect the Objective Contour view for as many pairs of free variables as you have time to analyze.
The following example shows the algorithm has found a local maximum (marked by the orange cross). You can see the global maximum for this optimization in the lower-right corner of the contour plot.
A constraint, such as a table gradient constraint, could cause a local maximum result. This result could be desirable, however, because it may be preferable for table smoothness to find a local maximum with a slight loss of torque compared to the global maximum (in this case, about 1.3 NM of torque (1%) which is within model accuracy).
To inspect contour plots for many pairs of free variables, you can configure the optimization output view to display multiple contour plots simultaneously. Simultaneous display can help locate those runs that have converged to a local optimum.
The following figure simultaneously shows contour plots for all pairs of free variables for the gasoline case study.
Inspect the Objective Graphs and Contour Views to check for optimizations that have terminated early. Early termination typically occurs with runs that have warning orange triangle Accept icons, but can also occur when the optimizer has returned a successful green square Accept icon.
The following figure shows an optimization run with a warning orange triangle Accept icon that has been forced to terminate because it exceeded the iterations limit.
In this case, the optimizer has almost found the optimal solution for this run. If this optimizer has taken a long time to run, then as this solution is almost optimal it is probably worth marking as acceptable (select the Accept box in the Optimization Results table for this run).
The following figure shows another example where an optimization terminated early because it exceeded the iterations limit.
In this case, the problem appears to be over constrained because the plots are entirely shaded yellow. You can check the constraint summary table or the output table to identify if constraints are met. Also inspect the constraint summary and constraint graphs.
Note Solutions on the constraint boundary and table gradient constraints often cause all objective and contour plots to be yellow (see Table Gradient Constraint Output). |
The constraint graphs for this case are shown in the following figure.
These constraint views confirm that Constraint2 is violated for this run. Therefore, this solution is probably best left as unacceptable. In cases like this, if it is not already marked as unacceptable, clear the Accept box in the Optimization Results table for this run.
The following figure shows an optimization that appears to have terminated early despite returning a positive exit flag. You can see that the optimizer has not located the maximum. You should investigate cases like this.
There are many reasons why an optimization appears to terminate early. Two common causes and possible resolutions are discussed in this section.
Foptcon may not return a local optimum if the following parameter values are too high:
Variable tolerance
Function tolerance
Constraint tolerance
In this case try reducing the values of these parameters to improve performance. However, do not reduce these parameter values too low (less than ~10-10) to avoid internal issues with foptcon. Models that have nonphysical nonlinearity can also cause failure.
Some nongradient-based algorithms may not return an optimum solution. An example of this is the genetic algorithm (ga) optimization in CAGE. A poor choice of parameters for such algorithms can lead to early termination of the optimization. For example, setting the Crossover Fraction parameter of the ga algorithm to 1 can lead to a situation where the algorithm prematurely converges. In this case, try rerunning the optimization at alternative parameter settings. For best results, rerun the algorithm with a Crossover Fraction lower than 1 (the default is 0.8).
Optimizations can terminate early because the models are noisy and you used a gradient based algorithm (foptcon) to solve the optimization problem.
If the contour plots or any results are suspicious you should always investigate model trends to check if they are sensible and not overfitting. Examine models in the CAGE Surface Viewer or the Model Browser response surface view. You may need to remodel.
To check whether your model is noisy, zoom in on a line plot of the model in the CAGE Surface viewer. Following is a plot of Objective1 against x around the value of x returned by the optimizer.
You can see that the model is noisy and the optimizer has (correctly) returned a local maximum of the model. However, this result is a maximum of the noise component in the model and not the physical component. If the noise is not behavior of the physical system, then you should remodel the noisy models in the Model Browser. The CAGE Import tool can be used to replace the noisy models with the results of the remodeling and the optimization can be rerun.
Functions that are flat in the vicinity of their optima can
be difficult to optimize. The following figure shows an example of
such a function,
, and its surface
plot.
This function has a global minimum at (0, 0) and is very flat in the vicinity of the optimal solution.
Using the foptcon algorithm in CAGE to find
the minimum of this function (from initial conditions of
) produces the result shown
in the following figure. The optimizer finds a solution at
, which is not optimal. In the
following plots, you can clearly see that the optimizer has not located
the minimum at (0, 0).
To adjust the optimizer to find the minimum, you can take one of several approaches:
Change the initial conditions.
For a gradient-based algorithm (foptcon in CAGE), changing the initial conditions can help the optimizer locate a minimum where the objective function is flat in the vicinity of the minimum. In the example shown in the previous figure, changing the initial conditions to (x,y) = (1,1) leads to foptcon finding the minimum at (0, 0).
Rescale the objective function.
Rescale the objective function with an operation that does not
change the location of any optimal solutions, e.g., try taking a square
root, fourth root or log, or multiplying by a positive scalar. Check
that the position of the optimum is not changed. When an objective
function is flat in the vicinity of an optimum, rescaling the objective
function can help gradient-based optimization algorithms such as foptcon in
CAGE. In the example shown in the previous figure, when foptcon in
CAGE is used to minimize
, the minimum at
(0, 0) is located.
Use a non-gradient based algorithm.
Try either the pattern search or genetic algorithm options. As these algorithms do not use gradient information, they can perform better when used on optimization problems with flat minima. In the example shown in the previous figure, the pattern search algorithm in CAGE located the minimum using the default settings.
Run the optimization from several initial condition values.
If you are using foptcon then another possible workaround is to set the Number of Start Points parameter to be greater than 1. This setting runs foptcon the specified number of times from different start conditions. Use this option only for the affected runs as it can be time consuming.
Change tolerances.
For a gradient-based algorithm (foptcon in CAGE), changing the variable or function tolerances can help the optimizer locate a minimum where the objective function is flat in the vicinity of the minimum. Reducing the variable and function tolerances may improve the convergence to the optimum value in this case.
 | Viewing Your Optimization Results | Tools for Optimizations With Multiple Solutions |  |
Includes the most popular MATLAB recorded presentations with Q&A sessions led by MATLAB experts.
| © 1984-2012- The MathWorks, Inc. - Site Help - Patents - Trademarks - Privacy Policy - Preventing Piracy - RSS |
