How to re-train a model optimized by Bayesian optimization on new data?

Hi,

I optimized a regression ensemble

model = fitrensemble(X,Y,...);

with active HyperparameterOptimizationOptions, so the resulting model object of type RegressionBaggedEnsemble contains a HyperparameterOptimizationResults object. How can I use this easily to re-train the model on a new data set with the best point found by the hyperparameter optimization algorithm? Something like this would be nice:

newModel = fitrensemble(Xnew,YNew,model.HyperparameterOptimizationResults.bestPoint);

My current approach is tedious, because I have to distinguish between the method the optimization algorithm selected (Bag,LSBoost) and set all parameters manually (and I'm even not sure if this is correct):

best = model.HyperparameterOptimizationResults.bestPoint;
ttmp = templateTree('MinLeafSize',best.MinLeafSize,'MaxNumSplits',...
    best.MaxNumSplits,'NumVariablesToSample',best.NumVariablesToSample);
if best.Method=='Bag' %#ok<BDSCA>
    newModel = fitrensemble(Xnew,YNew,...
        'Method','Bag','Learners',ttmp,'NumLearningCycles',best.NumLearningCycles);
else
    newModel = fitrensemble(Xnew,YNew,...
        'Method','LSBoost','Learners',ttmp,'NumLearningCycles',best.NumLearningCycles,'LearnRate',best.LearnRate);
end

This question is not restricted to fitrensemble, but includes all similar model functions available in Statistics and Machine Learning Toolbox (fitrlinear, fitrgp ...).

Answer 1

[Don Mathis]

Your current approach (explicitly passing the values that the optimization found) is the right way to do it.

There is a faster/simpler way which uses undocumented functionality, and which therefore may change in the future. So, use at your own risk. You create a template from your model and then fit it:

model = fitrensemble(X,Y,...);
tmp = classreg.learning.FitTemplate.makeFromModelParams(model.ModelParameters);
newModel = fit(tmp,Xnew,Ynew)