GPC is the C clipping library behind PolygonClipper. What you are experiencing is not due to a shortcoming of MATLAB or Hölz' MEX code, but is instead due to a known limitation of the underlying clipping algorithm implementation.
I had the same problem you are having (I only encountered it in certain cases), and I fixed it by snapping all polygon vertices to a grid before clipping, as Murta suggests.
Btw, thank you Sebastian Hölz, you have saved me the trouble of trying to write my own GPC interface!
