New users and old of optimization in MATLAB will find useful tips and tricks in this document, as well as examples one can use as templates for their own problems.
Use this tool by editing the file optimtips.m, then execute blocks of code in cell mode from the editor, or best, publish the file to HTML. Copy and paste also works of course.
Some readers may find this tool valuable if only for the function pleas - a partitioned least squares solver based on lsqnonlin.
This is a work in progress, as I fully expect to add new topics as I think of them or as suggestions are made. Suggestions for topics I've missed are welcome, as are corrections of my probable numerous errors. The topics currently covered are listed below.
1. Linear regression basics in matlab
2. Polynomial regression models
3. Weighted regression models
4. Robust estimation
5. Ridge regression
6. Transforming a nonlinear problem to linearity
7. Sums of exponentials
8. Poor starting values
9. Before you have a problem
10. Tolerances & stopping criteria
11. Common optimization problems & mistakes
12. Partitioned least squares estimation
13. Errors in variables regression
14. Passing extra information/variables into an optimization
15. Minimizing the sum of absolute deviations
16. Minimize the maximum absolute deviation
17. Batching small problems into large problems
18. Global solutions & domains of attraction
19. Bound constrained problems
20. Inclusive versus exclusive bound constraints
21. Mixed integer/discrete problems
22. Understanding how they work
23. Wrapping an optimizer around quad
24. Graphical tools for understanding sets of nonlinear equations
25. Optimizing non-smooth or stochastic functions
26. Linear equality constraints
27. Sums of squares surfaces and the geometry of a regression
28. Confidence limits on a regression model
29. Confidence limits on the parameters in a nonlinear regression
30. Quadprog example, unrounding a curve
32. Estimation of the parameters of an implicit function
33. Robust fitting schemes
35. Orthogonal polynomial regression
36. Potential topics to be added or expanded in the (near) future
Thanks for the examples. I have a suggestion on orthogonal polynomial fitting. Forsythe suggests a way to solve for both the correct orthogonal polynomials to use and the coefficients. The advantage is the full normal equation never has to be solved. Both the orthogonal polynomials and there coefficients are solved. It seems like an excellent method of data fitting.
Forsythe, George E. "Generation and use of orthogonal polynomials for data-fitting with a digital computer." Journal of the Society for Industrial & Applied Mathematics 5.2 (1957): 74-88.
I need robust regression methods in my diploma thesis and this work gives a verry good first impression of regression in matlab.
06 Dec 2007
20 Nov 2007
10 Oct 2007
07 Sep 2007
I loved this the most:
"% Likewise, reducing the value of TolFun need not reduce the error
% of the fit. If an optimizer has converged to its global optimum,
% reducing these tolerances cannot produce a better fit. Blood cannot
% be obtained from a rock, no matter how hard one squeezes. The rock
% may become bloody, but the blood came from your own hand."
09 Aug 2007
I am thai,who love Matlab.thank a lot.
12 Jul 2007
02 Jul 2007
Thanks very much!
12 Jun 2007
Good thank you
07 Apr 2007
13 Mar 2007
08 Feb 2007
10 Jan 2007
I'll see if I can do something with stochastic optimizers. It is a topic I apparently forgot to cover. Of course, the GADS toolbox is available for genetic algorithms. Please check back in a week or two.
10 Jan 2007
John, could you talk more about simulated annealing and other similiar optimization techniques? Or write a general function as you have done for gridfit. Thank you. I always learnt very much from you.
03 Jan 2007
28 Dec 2006
Great work! I found the linprog examples for L1 and L_infty regression quite helpful.
05 Dec 2006
20 Nov 2006
wilmer salazar trujillo
Deben promoverla con mayor intensidad en centros educativos desde primeros niveles
31 Jul 2006
not perfet though, it has usefull informations, but not many to explore,
overall its a good website
28 Jul 2006
It is very helpful for me to solve my work
27 Jul 2006
This package is very useful for me. It is excellent. Thank you for your help!
03 Jun 2006
Sir, it's a Excellent package. You should publish a book and please make sure that general students like me from India can buy it. Thnks for helping.
27 May 2006
14 May 2006
Sung SOo Kim
This is an excellent package.
Thank you so much.
08 May 2006
03 May 2006
21 Feb 2006
Thank you, i found it very useful
06 Dec 2005
A must read to beginners like myself. Great work that really helps - not like to on-line help of Matlab.
03 Nov 2005
31 Oct 2005
An excellent reource. You should publish this as a book, it would be a valuable resource for post graduates and carrer professionals! Really improved my routines by awnsering a lot of technical questions about using the optim toolbox (generally not covered in help or other books more general to the subject area). THANKYOU!
05 Oct 2005
13 Dec 2005
Six new topics have been added, some existing
topics expanded. Added titles and axis labels for all