As far as I know, no. There isn't a proper way to do that in manual mode. See the answer here:
However, if your contour plot is simple (not contourf; not a contour plot over a pcolor underlay), you might have a workaround. So long as the background color is solid, you can put matting under the text objects. It's a pain, but it can be done:
[c hc] = contour(x,y,z); hold on
gob = findobj(gca,'type','text');
axar = get(gca,'plotboxaspectratio');
R = [cosd(th) -sind(th); sind(th) cosd(th)].*[1; axar(1)];
xy = [p(1)+[0 p(3) p(3) 0 0]; p(2)+[0 0 p(4) p(4) 0]].';
boxc = min(xy,[],1)+range(xy,1)/2;
xy = (R*(xy-boxc).').'+ boxc;
hp = patch(xy(:,1),xy(:,2),'w');
This shows the patch extents after they've been rotated and scaled to fit. Setting the edge color makes them blend in.
This will attempt to transform the patch coordinates to correctly fit the text label for its given rotation and axes aspect ratio. If the figure window gets reshaped, the patches may be skewed problematically.
EDIT: You might be able to stretch this a bit more toward usage with contourf() or other types of configurations if you're willing to complicate things further. In this example, I'm doing contourf() with an explicit colormap matched to the number of contour levels for sake of simplicity. Instead of using a patch for matting, I'm using a fat line object. I calculate two additional points which are also transformed. These give sample points which should straddle the contour line. This allows us to know which contour level the label is on and select one of the neighboring colors from the color table.
[c hc] = contourf(x,y,z,nlevels-1); hold on
gob = findobj(gca,'type','text');
axar = get(gca,'plotboxaspectratio');
llist = get(hc,'levellist');
R = [cosd(th) -sind(th); sind(th) cosd(th)].*[1; axar(1)];
xy = [p(1)+[0 p(3)*[1 0.5 0.5]]; ...
[[1 1]*(p(2)+p(4)/2) p(2)+p(4)*[0.2 0.8]]].';
boxc = min(xy,[],1)+range(xy,1)/2;
xy = (R*(xy-boxc).').'+ boxc;
zsamp = [z(samp(1,1),samp(1,2)) z(samp(2,1),samp(2,2))];
[~,idx] = min(abs(mean(zsamp)-llist));
hp = plot(xy(1:2,1),xy(1:2,2),'color',levcol,'linewidth',5);
Again, show the matting and sample points for clarity. The figure on the right is the result.