How to interpret these Linear Mixed Model results?

Hello again!

I'm running a Linear Mixed Model using the fitlme function.

I'll try to explain quickly the variables. I'm testing each participant two times (two conditions - N.B.: it's not a longitudinal study, each subject is just tested 2 times in 2 different conditions). The response variable, named sigma, should depend on the fixed factor, named ecc_bins (shortly, it's a vector of different visual degrees). Teorically, the bigger the visual degree the bigger the sigma and I need to check if this relation (visual degree and sigma) changes - and how - in the two conditions.

Variables and Formula:

• Y variable (dependent, response variable) is called sigma (floats);
• Fixed effect (independent variable) called ecc_bins (from 1 to 10) - each sigma is associated with a specific bin;
• Random effect that is the participant (participant's numeric code).
• Grouping variable is the condition (integer 1 or 2).

As I found on the model page:

y ~ fixed + (random1|grouping1) + ... + (randomR|groupingR)

I wrote it like the following

lme = fitlme(tbl, 'sigma ~ ecc_bins + (participant | condition)');

Then, the result gives me

Fixed effects coefficients (95% CIs):
    Name                 Estimate    SE           tStat     DF            pValue    Lower      Upper  
    '(Intercept)'         1.0459     0.0078397    133.41    1.6148e+05    0          1.0305     1.0613
    'ecc_bins'            0.79073    0.0019764    400.09    1.6148e+05    0          0.78686    0.79461

Random effects covariance parameters (95% CIs):
Group: condition (2 Levels)
    Name1                Name2                Type          Estimate    Lower    Upper
    '(Intercept)'        '(Intercept)'        'std'         0.26803     NaN      NaN  
    'participant'        '(Intercept)'        'corr'             -1     NaN      NaN  
    'participant'        'participant'        'std'         0.01971     NaN      NaN  

I have a problem in interpreting these results.

1) I can see the significant p value for the fixed effect "ecc_bins" but what if I'd like to check which bin is significant and which not (post hoc?)? And why the lower and higher values for the random factor are NaN?

Moreover, when I check the random effects of the model

    [B, Bnames, stats] = randomEffects(lme)

I got something like:

     Random effect coefficients: DFMethod = 'Residual', Alpha = 0.05

    Group              Level      Name                 Estimate    SEPred       tStat      DF            pValue        Lower        Upper   
    'condition'        '1'        '(Intercept)'        -0.34986      0.01594    -21.948    1.6148e+05             0     -0.38111    -0.31862
    'condition'        '1'        'participant'        0.025728    0.0011722     21.948    1.6148e+05             0     0.023431    0.028026
    'condition'        '2'        '(Intercept)'        -0.14421     0.017638    -8.1763    1.6148e+05    4.4409e-16     -0.17878    -0.10964
    'condition'        '2'        'participant'        0.010605     0.001297     8.1763    1.6148e+05    4.4409e-16    0.0080628    0.013147

The low p values, in this case, mean that the 2 conditions are significantly different one from the other?

I'm really trying all the possibilities. I'm open to the chance that I've completely messed up the model.

Thank you for your help!

[Jakob B. Nielsen]

As to the accuracy of the model, I really can't say without having the data you are fitting. However, conclusion-wise you seem to be correct; a low pValue is indicative of that factor being significant. However, don't be trapped into thinking "different from the other" with regards to your interpretation!

Always think of the null hypothesis, which you set up for every term you test. "Condition 1 participant has a slope of 0". That is your null hypothesis for condition 1 participant, and you formulate similar null hypothesis for all other terms. You see a pValue that is 0, a tStat that is WAY away from 0, and you see furthermore that your Lower and Upper bounds for the Estimate is 0,0234 to 0,028. In other words: the confidence bounds on your estimate does not contain 0. And since your null hypothesis states that your condition 1 participant slope is 0 you can say, with confidence, that your null hypothesis is not true in this case. So in translation: participant 1 variation has a significant effect on your y. It is significantly different from 0.

Whether condition 1 and 2 are different from each other, the pValue does not tell you this. pValue only tells you if it is significantly different from 0. To see if condition 1 and 2 are significantly different from each other, you need to go and compare the Lower and Upper bounds on the Estimate. In your case, here, they indeed do not overlap.

[NMarco]

Thank you @Jakob B. Nielsen, your reply is helpful!

So if I understood correctly, in this case the null hypotesis would mean that the slope runs along the x axis. If the null hypotesis would be rejected (p < .05), the slope would have some degree of gradient (negative or positive). That seems my case here.

I'm not sure about something, though.

You said that:

participant 1 variation has a significant effect on your y.

I'm not sure I understood that. I do have a condition 1 (and 2), but here you cited participant 1. What does it mean? Being the participants the random effect I didn't expect they had an effect on Y.

Probably I've not 100% clear this part:

    Group              Level      Name               
    'condition'        '1'        '(Intercept)' 
    'condition'        '1'        'participant' 
    'condition'        '2'        '(Intercept)' 
    'condition'        '2'        'participant'

Is this some kind of comparison? Like the effect of condition 1 (and 2) on the participants?

[NMarco]

Moreover, when you say

you need to go and compare the Lower and Upper bounds on the Estimate. In your case, here, they indeed do not overlap.

you mean, take the estimate, the lower and the upper values for condition 1 row and condition 2 row

        Group         Level           Name      Estimate    Lower        Upper   
    'condition'        '1'        'participant' 0.025728   0.023431     0.028026
    'condition'        '2'        'participant' 0.010605   0.0080628    0.013147

and check if they overlap each other.

In my case, condition 1 goes from 0.023 to 0.028 while condition 2 goes from 0.008 to 0.013.

They definetely do not intersecate. So I can say that the 2 conditions have different effect on the responses (?)

[Jakob B. Nielsen]

Yes, it is condition 1 that I should have written. It is because it has "participant" as the name, but you are correct.

They definetely do not intersecate. So I can say that the 2 conditions have different effect on the responses (?)

Yes, that would be my interpretation. Both conditions have a significant effect, but the effect is also significantly different from each other.

[NMarco]

That's wonderful! Thank you so much!

Lastly, what if I want to check the effect of each visual degree bin (the fixed effect)? Because the response variable (in my case is named sigma) should differ depending on the ecc_bins value.

[Jeff Miller]

Doesn't this model

lme = fitlme(tbl, 'sigma ~ ecc_bins + (participant | condition)');

imply that the different conditions have different participants (e.g., think of condition like 'gender')? If I understood your description of the setup, though, you have the same participants in both conditions, so this wouldn't be the right model.

[NMarco]

@Jeff Miller, yes, each participant underwent to both conditions. I thought I organized correctly the table. Inside the "participant" column each individual is repeated twice and has data from both the conditions.

I.e.: each participant has his own "sigma" data, "bins", two times, one for each condition.

Below it's just an example:

sigma   bins    condition   participant
0.234   1           1           2
1.998   2           1           2
4.231   3           1           2
1.241   1           2           2
2.223   2           2           2
3.234   3           2           2

That's what is bothering me most. I'm exploring several possibilities in arranging the formula for the model.

[Jeff Miller]

That looks to me like an ANOVA design with 2 repeated-measures factors (bins*condition). If participant is the only random factor, then I think LME is overkill.

[NMarco]

Thanks for this reply, in fact in these last days I opted for ANOVA. I'll see how it is going.

Thanks!!