How can I compute regression coefficients for three or more predictor variables?

How can I obtain the coefficients (b) of multiple linear regression between mean free joint angles (ankle, hip, knee angles time-normalised to 101 data points) as predictor variables (X = x1, x2, x3) and changes in the mean-free foot COM as an outcome variable (y)

Answer 1

[Star Strider]

Assuming your expression is:

y = X * B

where B is the parameter vector, the easiest way to calculate B is:

B = X\y;

Also see regress, since it provides statistics on the fit and confidence intervals on the estimated parameters.

[Tim Bennett]

Thanks for you help.

I've just entered the following into Matlab:

X = [ones(size(x1)x1 x2 x3];

B = X\y

where the values of y, x1, x2, and x3 are 1 x 20 each. However the B output is giving me a 80 x 20 matrix filled with zeros apart from row 44 that contains a range of values (e.g 0.0006 0.0033 -0.002 etc). Are these the coeffiecents as I was expecting a single vector column of coefficients?

[Star Strider]

It's my pleasure to help!

You need to transpose x1, x2, and x3 to be [20 x 1] to form your X matrix:

X = [ones(length(x1),1)  x1'  x2'  x3'];

and if y is not also already a column vector, it needs to be. Assuming y is currently a row vector as well, your multiple linear regression parameter estimation statement becomes:

B = X\y';

That should work.

That the B you calculated is an [80 x 20] matrix filled with zeros results from your current X (not the one I constructed here) being an 80-element row vector rather than the [20 x 4] matrix you intended.

[Tim Bennett]

Thanks again for your help. Your code worked fine giving me a [4 x 1] vector of the coeffient estimates.

If I was to add another output variable to the equation (i.e. adding data from the y-axis or the position of the foot to the data from the original x-axis of the position of the foot) would I then enter the data as y1 + y2 = y and use the same B = X\y'; equation to give me the coefficient estimates in a [2 x 3 ] matrix (i.e. 2 columns for the x and y axis and 3 rows for the hip, knee and ankle joints)?

Sorry if this doesn't make much sense.

[Star Strider]

If I understand you correctly, what you want to do is indeed possible. The easiest way to find out for sure is to experiment, so I did. I also want to be sure that I'm addressing the issue you raised:

X = [rand(10,3)];                           % Original Data
B = [[1; 3; 5]  [2; 4; 6]];                 % Parameters
Y = [X*B(:,1)  X*B(:,2)];                   % Calculate Y
X1 = [ones(10,1) X+randn(size(X))/10000];   % Add ‘randn’ noise to simulate reality
Best = X1\Y                                 % Estimate parameters

It gave the following:

Best =

     368.4308e-006   242.2789e-006
     999.3107e-003     1.9993e+000
       2.9999e+000     3.9999e+000
       5.0002e+000     6.0005e+000

It works. (You don't have to add noise to the simulation, but I did in order to be sure it worked.) Best is the estimated parameter matrix, B.

Thank you for accepting my answer!

[Tim Bennett]

Thanks for getting back to me. I just need a bit of help breaking this down.

(1) I have a [20 x 3] matrix of predictor variable data (X) (i.e. column 1 = mean free hip angles, column 2 = mean free knee angles etc) and a [20 x 2] of outcome variable data (Y) (i.e. column 1 = mean free foot centre of mass in the x-axis and column 2 = mean free foot centre of mass in the y-axis). Therefore, do I need to calculate Y when the data is available or still use the equation Y = [X*B(:,1) X*B(:,2)];? I'm just trying to understand how the Y and X variables are linked together using this code to produce the estimated coefficients?

(2) Where you have 'B = [[1; 3; 5] [2; 4; 6]];' would the equation B = X\Y'; not automatically give the coefficient estimates (B)in a [2 x 3 ] matrix?

(3) Do I still need to calculate X using the following equation:

X = [ones(length(x1),1) x1' x2' x3'];

I'm trying to compare the multiple linear regression coefficients with the output from a geometric model that linked the hip to the foot centre of mass giving the following values in a [2 x 3] matrix:

-0.1008, 0.1717, -0.0103

-0.7806, -0.4124 , -0.0788

where the first and second rows consisted of partial derivatives of the hip, knee, and ankle in relation to the X and Y positions of the COM of the foot respectively.

[Star Strider]

As I mentioned in my response in your other thread —

1. I created my example code as an illustration that you could indeed have two dependent variables and regress them against your independent variables. You do not have to calculate Y if you have already defined it with your data as your matrix of dependent variables.

2. It should give you your coefficients in a [4 x 2] matrix as I have it coded here.

3. Yes.

Multiple linear regression is appropriate if your model assumes a linear relationship between the independent and dependent variables. If that assumption is not valid, you will likely have to use a nonlinear regression model.