Smooth interpolation for warping an image using a deformation map

I have a series of 2D images that I need to register similar to the one below.

The registration is neither geometric nor rigid, but defined by a displacement map. I derive the displacement map by computing the gradient of a spatial registration error map (called “errorMap” below).

I know that errorMap is correct and exactly describes the spatial registration error. I use errorMap to compute the deformation map by computing its gradient.

Then I apply this deformation map to the reference image using imwarp. The code is:

[uy, ux] = gradient(errorMap);
D = cat(3, uy, yx);
B = imwarp(im, D, 'linear');

The problem I’m running into is that, when I apply the displacement map to the image, I get, well, this:

I feel like it’s moving the pixels from the reference image into the source image rather than interpolating smoothly, leaving very conspicuous discontinuities at the edges. This is regardless of the interpolation method I apply. I tried applying the deformation map in steps of 1/10 to try to "nudge" it towards the final registration:

B2 = imwarp(im, D./10, 'linear');
for idx = 1:9
    B2 = imwarp(B2, D./10, 'linear'); 
end

While this removes some of the discontinuities, it also leaves what seem to be blotchy smoothing artifacts:

It makes no difference whether I use a linear or a cubic interpolation. Using a nearest neighbor interpolation doesn't work well at all, especially if I "nudge" the deformation by very small amounts (essentially, no deformation is applied in this case).

How can I apply this deformation map in a smooth manner, and with sub-pixel accuracy? What options are available in Matlab and the image processing toolbox? Is there an alternative to imwarp that would do a better job of applying this deformation map?