How can I do robust regression in a step-wise regression?

I am doing step-wise regression to address multicollinearity in my data. However, I am thinking of using robust regression in a stepwise regression. In the matlab help,it is clearly mentioned as a tip that "You cannot use robust regression with stepwise regression.Check your data for outliers before using LinearModel.stepwise." Is there a technical reason why one cannot do robust regression in stepwise regression. If no, could you please help me to do the same.

Answer 1

[Jyotish Robin]

Hi Vivek,

why doing robust regression in stepwise regression is not advisable??

Robust regression works by doing iteratively reweighted least squares fits. The coefficient estimates are the same as weighted least squares estimates using the final robust weights, but the coefficient covariance is calculated differently for robust fits compared to weighted fits.

Suppose we fit two models M1 and M2 such that M2 has 1 extra predictor compared to M1. To do "LinearModel.stepwise" with robust fitting, we would need a way to statistically compare M1 and M2 accounting for the fact that the final robust weights for M1 and M2 are different. This may be doable either exactly or with some approximations (e.g., assuming final robust weights don’t change between the two fits).

Maybe you could consider the following two alternatives:

1. As an approximation to "stepwiselm" with robust fitting, we could use "stepwiselm" with robust weights from a full model robust fit. Then use robust fitting on the model selected by previous "stepwiselm" to get standard errors based on robust covariance.

Attached you can find an example for this.

2. We could use "sequentialfs" along with mean squared error from a robust fit as the “criterion” to approximate the "stepwiselm" predictor selection process.

Note: Linear.stepwise and stepwiselm are functionally similar

Hope the above suggestions are helpful!