The problem is, you don't really want an envelope.
For example, here are a couple of fits that will produce an envelope.
slm_upper = slmengine(x,y,'env','sup','plot','on','knots',30);
slm_lower = slmengine(x,y,'env','inf','plot','on','knots',30);
Those are envelope fits, i.e., least upper bound and greatest lower bound functions.
But what you have drawn are functions that sort of look like that, but go where you want them to go, ignoring some of your data. So they skip some points that you consider outliers. You want maybe a function that can intelligently (defined by what you consider an outlier) exclude perhaps some 1 to 5% of the points as outliers.
The problem is, the eye is good at seeing a pattern that it likes. The computer, not so good. Computers do what they are programmed to do.
Perhaps the best suggestion I can offer is to use a tool like my SLM as below in a multiple step process:
slm_upper0 = slmengine(x,y,'env','sup','plot','on','knots',30);
res_upper = slmeval(x,slm_upper,0) - y;
keep_upper = find(res_upper > tol);
drop_upper = find(res_upper <= tol);
slm_upper = slmengine(x(keep_upper),y(keep_upper),'env','sup','plot','on','knots',30);
Thus, find the points that were on the boundary in one envelope, then exclude them form the fit, and repeat the fit. Do a similar procedure for a lower semi-envelope. I'm not sure I see a better way.