In regards to your question whether one can use min() or max() in writing their nonlinear constraints, the answer is yes.
A possible reason for why your constraint is ignored would be that the solver doesn't know about it.
I assume that you have implemented your nonlinear constraint function (say function [c, ceq] = myconst(x)) and your problem structure has a field called 'nonlcon' and its value is 'myconst'. As an example, using createOptimProblem
problem = createOptimProblem('fmincon','x0',randn(2,1), 'objective',myobjfun,'nonlcon',myconst,'lb',[-2;-2],'ub',[2;2],'options',opts);
you had your problem, that was fed into GlobalSearch as:
gs = GlobalSearch;
[x,f] = run(gs,problem)
Could you confirm that you have (more or less) done this?
P.S.: I think you can use "X(20) <= 100 - max([X(1:3) X(5:7) X(10:17)])" instead of "X(20) <= 100 - max([max(X(1:3)) max(X(5:7)) max(X(10:17))])".