|
"Jay " <aro-_-mail@web.de> wrote in message
news:ktdike$j23$1@newscl01ah.mathworks.com...
> Hi!
>
> There is a thorough description of decision forests in this report
> http://research.microsoft.com/apps/pubs/default.aspx?id=155552
> and I am looking to use a framework like this in Matlab.
> Matlabs Treebagger class in combination with the classregtree seems the
> way to go but I have two problems with this solution:
> 1. The splitcriterion (In the above report called the weak learner model)
> seems to be fixed. classregtree always aims to minimize a least square
> error in every node. It would be nice if I was able to write the weak
> learner model myself and to decide how to split.
>
> 2. What exactly is the predictor model used in classregtree, i.e., what is
> the function a leaf node uses to make a prediction, and how does
> treebagger handle the different outputs of its single trees? It would also
> be nice if I could change the predictor model used.
>
>
> So, is there a way to do what I want Matlab (either with Treebagger or
> with another implementation)? I would highly appreciate not having to
> modify any C++ files or alike :)
>
It's unlikely someone in this forum will want to read a ~140 page document
you are referring to. Without reading it, I'd like to clarify a few things.
"Weak learner" in the ensemble literature refers not to the split criterion
imposed by a decision tree, but to any data-fitting model model used
repeatedly for growing an ensemble of such models. The fitensemble function
provides several ensemble-learning algorithms. Some of them use decision
tree as the weak learner, and some of them use k-NN and discriminant.
The classregtree class, as well as the decision tree classes introduced
later (see Alan's response), support several split criteria for
classification and one (MSE) for regression. Even if you focus on
regression, decision tree is not fixed. An (perhaps, the most) important
tuning knob in ensemble learning by decision trees is the tree size. You can
control it by passing the 'minleaf' and 'minparent' parameters to TreeBagger
or to fitensemble (via a learner template).
At present, we do not support arbitrary weak learners for ensembles out of
the box. But if you want to code a weak learner model yourself, feel free to
get in touch with me, and I'll see what can be arranged.
Ilya
|