MSE between a set of data points (x,y) and a defined line

Hi all,

I'm a psychologist and i'm totally new to Matlab. I'm trying to reproduce an analysis reported in a scietific paper where the researcher compared a curve drawn by participants with an ideal straight line using a formula representing the square of the root mean square (in the attachment the piece of the paper where the analysis is reported).

What i have now is a series of coordinates (a file .dat) divided in two columns (X,Y), representing what the subject has drawn (actual trajectory) with the mouse and i also have the two points where the ideal trajectory should be.

From the formula I understood that the RMS should be computed based on the defined integral (defined by the last X of the subject and the first X of the movement done by participant) of the square of the subtraction between the two functions (actual and ideal). My main problem is that i don't know what should i put as the argument of the integral (what should i subtract). Their results of the formula are always oscillating between 0.0001 and 0.05.

I've tried a simple piece of code (below) where i tried to subtract the integrals of the two curves (i guess basically subtracting the areas) but i really don't think is what i should be doing here.

Thank you in advance for your help,

Alessandro

% % % % % % % % % %  
% Lines comparisons% 
% % % % % % % % % % 

load my_xy.dat;     %  read actual trajectory data into the my_xy matrix
Xr = my_xy(:,1);    %  copy first column of my_xy into Xr(eal)
Yr = my_xy(:,2);    %  copy second column of my_xy into Yr(eal)
Xd = my_xy (:,3);
Yd = my_xy (:,4);

Xc = [0.0,503.46];%  Xc(orrect) ideal line x coordinates
Yc = [-352,151.46];%  Yc(orrect) ideal line y coordinates

plot(Xr,Yr,'--',Xc,Yc);     %  plot actual trajectory and ideal trajectory

% 1) trying to replicate the formula but not succeding
MSE = ((trapz(Xr,Yr)-trapz(Xc,Yc)).^2)/(Xr(end)-Xr(1));

[Image Analyst]

Can you make it easy for us by attaching some data file?

[Star Strider]

‘(in the attachment the piece of the paper where the analysis is reported)’

No attachment.

Is the method you want to implement described in the paper, or do you have a specific analysis method in mind? It is not clear (at least to me) what you want to do.

[Alessandro]

Thank you for the fast answer and sorry about the missing attachment, it should be showing now i hope. I'm attaching also my file.txt where i have two columns of values (respectively my X and Y coordinates) representing the movement of the mouse.

Answer 1

[Roger Stafford]

Before you carry out your integration it is necessary to establish the y-values for your ideal straight line at the corresponding x-values of the participant data. If this ideal line is defined as passing through the two points (0.0,-352) and (503.46,151.46), then its equation is:

   y = -352 + (151.46-(-352))/((503.46-0.0)*(x-0.0)

You called the x-values of the participant Xr and the participant y-values Yr. The ideal y-values at the corresponding points would therefore be:

   Yc = -352 + (151.46-(-352))/((503.46-0.0)*(Xr-0.0)

To obtain the trapezoidal approximation to the integral you described, it would be computed by:

   a2 = trapz(Xr,(Yr-Yc).^2)/(Xr(end)-Xr(1));

This is the mean square error. To get the root mean square error, take its square root.

Answer 2

[Star Strider]

The integral you posted is actually the Mean Square Error (MSE), not the RootMean Square Error (RMS, RMSE) which is the square root of the MSE.

While I was waiting for clarification, I coded the following code snippet:

Xc = [0.0,503.46];  %  Xc(orrect) ideal line x coordinates
Yc = [-352,151.46]; %  Yc(orrect) ideal line y coordinates

Xr = [0.1   498];   % Created Data
Yr = [-360  150];   % Created Data

Bc = [Xc' [1;1]]\Yc';    % Slope and Intercept of the ‘Correct’ Line
Br = [Xr' [1;1]]\Yr';    % Slope and Intercept of the ‘Real’ Line

Yp = [Xc' [1;1]]*Br;     % Projected Y-Coordinates of Yr w.r.t. Xc

Yd = Yc'-Yp;             % Difference of Y-Values at the Same Xc Values

MSE = @(Q) mean(Q.^2);          % Mean Square Error
RMS = @(Q) sqrt(mean(Q.^2));    % Root Mean Square Error

YdMSE = MSE(Yd)
YdRMS = RMS(Yd)

It takes beginning and ending points of both lines and calculates the slopes and intercepts of both in Bc and Br respectively. In order to get Y-values at common X-values ( Xc chosen for those ), it calculates projected coordinates of the Yr line based on Bc, as Yp. It then compares Yp and Yc as Yd and calculates the MSE and RMSE of Yd.

This seems to implement the idea in your attachment (at least as I read it), since it compares the differing Y-values at the same X-values.

Answer 3

[Alessandro]

Thank you for both answers, I'll try your suggestions right away!

Alessandro

Answer 4

[Alessandro]

Hi again,

I've tried both methods in the code written below:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
load my_xy.txt;     %  read data into the my_xy matrix
Xr = my_xy(:,1);     %  copy first column of my_xy into Xr
Yr = my_xy(:,2);     
Xc = [0.0;503.46];  %  Xc(orrect) ideal line x coordinates
Yc = [-352;151.46]; %  Yc(orrect) ideal line y coordinates

plot(Xr,Yr,'--',Xc,Yc);     %  plot 
axis([-600,600,-600,600]);

%1) Method number 1 results YdMSE = 2.417957095842531e+03, YdRMS = 49.172727154821615

Xr2 = [Xr(1); Xr(end)];   % Created Data
Yr2 = [Yr(1);  Yr(end)];   % Created Data
Bc = [Xc' [1 1]]\Yc';    % Slope and Intercept of the ‘Correct’ Line
Br = [Xr2' [1 1]]\Yr2';    % Slope and Intercept of the ‘Real’ Line
Yp = [Xc' [1 1]]*Br;     % Projected Y-Coordinates of Yr w.r.t. Xc
Yd = Yc'-Yp;             % Difference of Y-Values at the Same Xc Values
MSE = @(Q) mean(Q.^2);          % Mean Square Error
RMS = @(Q) sqrt(mean(Q.^2));    % Root Mean Square Error
YdMSE = MSE(Yd);
YdRMS = RMS(Yd);

%2) Method number 2 results a= 268.243455715561 , a2 = 7.195455153422637e+04

Yc2 = -352 + (151.46-(-352))/((503.46-0.0)*(Xr-0.0));
tYc2 = transpose(Yc2);% transposing Yc2 to perform the following subtraction 
a2 = trapz(Xr,(Yr-tYc2).^2)/(Xr(end)-Xr(1));
a =sqrt(a2); 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

For method 1, I just added in Yc and Xc the semicolons (;) in line 5 and 6 and eliminated the semicolons in Bc and Br between [1;1] beacuse matlab was giving me an error " CAT arguments dimensions are not consistent". For method 2, I just transposed the "ideal" y-values to let matlab do the subtraction in the successive line (it was giving me for "Yr-Yc2" the error: "Matrix dimensions must agree.").

I don't know if a2 and YdRMS should coincide since you proposed different methods but my main concern is that the results obtained in the paper with the "same" analysis are always 0.0... results (e.g. 0.002, 0.03 etc...) as shown in the image below taken from the paper.

I don't know if I made mistakes in translating your codes to my data (sorry in advance) but i'll give you more information from the experiment that i'm trying to replicate (cause maybe the error is in my conception of real and ideal data).

Basically participants are trying to connect two points (starting and target points) on the screen on the vertical midline, using a pen stylus on a tablet. The task is very easy but in some trials participants "actual" line will not be displayed but instead a distorted line (rotated by a certain number of degrees to the left)will be displayed (in my case the line was distorted by 45 degrees on the left). Participants will try, of course, to correct this distortion by applaying a corretion to their trajectory. In the end, the program i wrote will give me as output a txt file (the my_xy.txt file i attached in my second comment). To create the ideal line that would have perfectly corrected the distortion bias i created a line connecting two points: the first (0.0, -352) is where the stylus started and the last point is the point where is shluld have been (the target point) rotated by the opposite number of degrees of the distortion (in my case 45 degrees to the right) resulting in (503,151.46) (from this i obtained my Xc and Yc coordinates).

Thank you in advance for any suggestion Alessandro

[Star Strider]

My approach uses the beginning and ending points of your lines (‘correct’ and ‘real’), because that is how I understood your Question. I cannot comment on the results in the paper and your data, because I haven’t read the paper. (However, the paper’s definition of RMS is actually MSE, so there could be problems with its calculations.) I suggest you review how the paper defined its units, since that could explain the discrepancy between your values and those reported in the paper. (Did the authors of the paper normalise their data by some method we don’t know about, for instance?)

[Alessandro]

You are absolutely right, my coordinates represents pixels on the screen (where x = 0 and y = 0 is the middle of the screen) and are related to my resolution while maybe they have used mm or cm as units. The paper is in the attachment and the methods are in the last pages.

I'll try to convert my units. Thanks again for the suggestion! Alessandro

[Star Strider]

My pleasure!

In their Figure 2, they are clearly measuring the line length as 222 mm. Scaling your data to the same units should produce roughly the same results. Scaling it is probably as simple as measuring the screen and applying the appropriate scaling factor. I suggest you do this in both the horizontal and vertical directions to verify that the pixels are approximately square. If so, you need apply only one scaling factor of mm/pixel to get the same units as those in the paper.