Anova-N output question

Hello community!

The closest prompt I could find that is similar to this would be: https://www.mathworks.com/matlabcentral/answers/1876737-anova-n-outputs-as-not-full-rank-returns-nan-p-value?s_tid=sug_su , but the reason their's showed NaN was there was not enough values. For mine I have 5.4 thousand entries, so I'm not sure that is the problem.

To reduce clutter of the code, I am going to attach the .mat files and the one-line of code.

pTHalf = anovan(stats(:,1), {Patho CellLine MW},'model','interaction','varnames', ...
    {'Pathology','Cell Line','Molecular Weights'});

I ensured the data is of the same types allowed within the format of the Anova-n overview page. The only thing that I could think of was that there are more than two groups within the Cell Line and Molecular Weights groupings, but the only one that worked was the molecular weights, so I also doubt that is the reason.

This is the output below. Why do I have missing sections associated with a 'not full rank'. I do not see anything on the anova-N page that discusses this.

Thanks community!

Nick

Answer 1

[Divyam]

Hi @Nicholas Scott, it appears that the terms 'Pathology' and 'Cell Line' are highly correlated. In such scenarios, removing one of them should not affect the model. The solve this problem, simply remove the term with lower contribution.

To figure out which terms are contributing more, change the 'sstype' of your 'anovan' function to '1'. This will calculate each term's contribution by adding that term to the terms already included before it. For more information about the 'sstype' refer to the following documentation link: https://www.mathworks.com/help/stats/anovan.html#:~:text=single%20%7C%20double-,sstype,-%E2%80%94%20Type%20of

% To determine the effect of adding Cell Line 
pTHalf = anovan(stats(:,1), {Patho CellLine MW},'model','interaction','varnames', ...
    {'Pathology','Cell Line','Molecular Weights'}, 'sstype',1);
% To determine the effect of adding Pathology
pTHalf = anovan(stats(:,1), {CellLine Patho MW},'model','interaction','varnames', ...
    {'Cell Line','Pathology','Molecular Weights'}, 'sstype',1);

Upon running the above code, it is observed that the 'Pathology' term is not a full rank term, and its F value is far lower than the 'Cell Line' term. Hence removing the 'Pathology' term from the 'anovan' function should remove the issue.

pTHalf = anovan(stats(:,1), {CellLine MW},'model','interaction','varnames', ...
    {'Cell Line','Molecular Weights'}, 'sstype',1);

In the above results you can observe that the terms are full ranked and there are no non-full rank terms.

[Nicholas Scott]

Absolutely incredible! I'd love to know how you knew exactly how to trouble shoot that so I can be more effective at trouble shooting in the future! I now understand (shockingly) that two of my grouped variables were too correlated to be considered full rank. Even just doing a 2way anova between the two still provides that, now that I have checked! Crazy!

Thank you so much @Divyam

[Divyam]

When faced with an error message/warnings you should check out your hypothesis and then make the suitable changes.

In your case since the terms were not full rank, it could be deduced that a certain term is unable to determine the output with its combinations. We can rule out the 'MW' term since it is a full rank term so we look into the individual contributions of the 'Pathology' and 'Cell Line' terms to find the least contributing term and when we omit the 'Pathology' term we obtain good results which fit the data provided.

However in any sort of statistical analysis, it should be noted that good results are not indicative of the solution, you should find a reasonable reason as to why 'Cell Line' and 'MW' produce the best results and why 'Cell Line' and 'Pathology' are correlated, only then you can say that your model not only fits the data but is also logically sound.

[Nicholas Scott]

Thank you Divyam, I had actually started to ponder this question as to why the Line and pathology were tightly correlated and realized exactly where I went wrong with making this an anova3 way (at least with that categorization), so I change it to a different categorization to make more logical sense as to what might be helpful to deducing the behavior, trends, and interactions between variables. Thank you SO much!

[Divyam]

You are welcome @Nicholas Scott, all the best for your work!