File Exchange

image thumbnail

RegularizeData3D

version 1.1 (645 KB) by

Produces a smooth surface from scattered 3D input data.

28 Downloads

Updated

View License

RegularizeData3D is a modified version of GridFit from the Matlab File Exchange. RegularizeData3D does essentially the same thing, but is an attempt to overcome several shortcomings inherent in the design of the legacy code in GridFit.

* GridFit lacks cubic interpolation capabilities. Interpolation is necessary to map the scattered input data to locations on the output surface. The output surface is most likely nonlinear, so linear interpolation used in GridFit is a lousy approximation. Cubic interpolation accounts for surface curvature, which is especially beneficial when the output grid is coarse in x and y.

* GridFit's "smoothness" parameter was poorly defined and its methodology may have led to bad output data. In RegularizeData3D the smoothness parameter is actually the ratio of smoothness (flatness) to fidelity (goodness of fit) and is not affected by the resolution of the output grid.
 Smoothness = 100 gives 100 times as much weight to smoothness (and produces a nearly flat output surface)
 Smoothness = 1 gives equal weight to smoothness and fidelity (and results in noticeable smoothing)
 Smoothness = 0.01 gives 100 times as much weight to fitting the surface to the scattered input data (and results in very little smoothing)
 Smoothness = 0.001 is good for data with low noise. The input points nearly coincide with the output surface.

* GridFit didn't do a good job explaining what math it was doing; it just gave usage examples.
             
For a detailed explanation of "the math behind the magic" on a 3D dataset, see:
            http://mathformeremortals.wordpress.com/2013/07/22/introduction-to-regularizing-3d-data-part-1-of-2/

and to apply the same principles to a 2D dataset see:
            http://mathformeremortals.wordpress.com/2013/01/29/introduction-to-regularizing-with-2d-data-part-1-of-3/

Both of these links include Excel spreadsheets that break down the calculations step by step so that you can see how RegularizeData3D works. There are also very limited (and very slow!) Excel spreadsheet functions that do the same thing in 2D or 3D.

Comments and Ratings (9)

Serge

Serge (view profile)

@Andreas Almqvist "Gridfit is not an interpolant. Its goal is a smooth surface that approximates your data, but allows you to control the amount of smoothing." -Gridfit doc

But if your data does have noise, then try this with different 'smooth' settings:

load('zygo_data') %load X,Y,Zraw
Zgd = griddata(X(~isnan(Zraw)),Y(~isnan(Zraw)),Zraw(~isnan(Zraw)),X,Y); %griddata
Zrd = RegularizeData3D(X,Y,Zraw,unique(X(:)),unique(Y(:)),'smooth',0.00001); %RegularizeData3D
figure(1);surf(X,Y,Zraw),a=axis ;camlight;shading interp;title raw
figure(2);surf(X,Y,Zgd) ,axis(a);camlight,shading interp,title griddata
figure(3),surf(X,Y,Zrd) ,axis(a);camlight,shading interp,title RegularizeData3D

Andreas Almqvist

I thought this function would give a very similar result to what GRIDFIT give, for the same choice of smoothing. But for me both shape and detail seem very unrealistic and really far far off what MATLAB built-in scatteredInterpolant produce. For me GRIDFIT is the preffered choice.

I've copied in a MATLAB-script to that test this below. The surface data can be found here: https://www.dropbox.com/s/e9ac68n03i5lmj1/zygo_data.mat?dl=0

% Testing different ways of retrieving missing data from a 3D measurement
% of a cylindrically shaped surface with roughness

clear all;
%% Surface roughness data
load('zygo_data'); % loads X, Y and Zraw
% Visualise data
figure(1); set(gcf,'color','w')
surf(X/1e-6,Y/1e-6,Zraw/1e-6);
camlight right;
lighting phong;
shading interp;
title 'Zygo surface data';
set(gca,'fontsize',48)
xlabel(['[' 956 'm]'],'FontSize',32)
ylabel(['[' 956 'm]'],'FontSize',32)
zlabel(['[' 956 'm]'],'FontSize',32)

%% Turn the scanned point data into a surface using gridfit
% Data first needs to be reshaped into vectors
[M,N] = size(X);
x=reshape(X,N*M,1);
y=reshape(Y,N*M,1);
z=reshape(Zraw,N*M,1);

% Now create a grid onto which to interpolate the surface height data
xi = linspace(min(x),max(x),N);
yi = linspace(min(y),max(y),M);
[Xi,Yi] = meshgrid(xi,yi);

% Calling the gridfit function to recreate the zygo surface
tic;
Zgf = gridfit(x,y,z,xi,yi);
toc;
figure(2);
% Visualise the data
surf(Xi/1e-6,Yi/1e-6,Zgf/1e-6);
camlight right;
lighting phong;
shading interp;
title 'Recreate the surface using gridfit';
set(gca,'fontsize',48)
xlabel(['[' 956 'm]'],'FontSize',32)
ylabel(['[' 956 'm]'],'FontSize',32)
zlabel(['[' 956 'm]'],'FontSize',32)

%% Turn the scanned point data into a surface using RegularizeData3D
% Data first needs to be reshaped into vectors
[M,N] = size(X);
x=reshape(X,N*M,1);
y=reshape(Y,N*M,1);
z=reshape(Zraw,N*M,1);

% Now create a grid onto which to interpolate the surface height data
xi = linspace(min(x),max(x),N);
yi = linspace(min(y),max(y),M);
[Xi,Yi] = meshgrid(xi,yi);

% Calling the gridfit function to recreate the zygo surface
tic;
Zrdf = RegularizeData3D(x,y,z,xi,yi);
toc;
figure(21);
% Visualise the data
surf(Xi/1e-6,Yi/1e-6,Zrdf/1e-6);
camlight right;
lighting phong;
shading interp;
title 'Recreate the surface using gridfit';
set(gca,'fontsize',48)
xlabel(['[' 956 'm]'],'FontSize',32)
ylabel(['[' 956 'm]'],'FontSize',32)
zlabel(['[' 956 'm]'],'FontSize',32)

%% Turn the scanned point data into a surface using MATLAB built-in function scatteredInterpolant
% Remove NaNs
ind = find(~isnan(z));
xnn = x(ind);
ynn = y(ind);
znn = z(ind);

% Calling the built-in scatteredInterpolant function to recreate the zygo
% surface
tic;
F = scatteredInterpolant(xnn,ynn,znn);
Zsip = F(Xi,Yi);
toc;
figure(3);
surf(Xi/1e-6,Yi/1e-6,Zsip/1e-6);
camlight right;
lighting phong;
shading interp;
title({'Recreate the surface using scatteredInterpolant'});
set(gca,'fontsize',48)
xlabel(['[' 956 'm]'],'FontSize',32)
ylabel(['[' 956 'm]'],'FontSize',32)
zlabel(['[' 956 'm]'],'FontSize',32)

%% Producing the relative difference between the gridfit and the scatteredIterpolant surfaces
figure(4);
surf(Xi/1e-6,Yi/1e-6,(Zgf-Zsip)/max(abs(znn)));
camlight right;
lighting phong;
shading interp;
title({'The relative difference between the to surfaces';});
set(gca,'fontsize',48)
xlabel(['[' 956 'm]'],'FontSize',32)
ylabel(['[' 956 'm]'],'FontSize',32)
zlabel(['[' 956 'm]'],'FontSize',32)

Serge

Serge (view profile)

I use this iteratively: fit a "rubber sheet", remove outliers and repeat. Works amazing, accurate and fast!

Is it possible to use same methodology on 2D data, perhaps hack or trick this function? I don't wish to use excel... I like the options this gives for dealing with boundaries and I am not sure I can replicate this functionality on 2d data.

Matthew

Nice addition to GridFit. Appears well implemented and examples provided are very clear.

I have been using this almost a week everyday. I work with a lot of tables of data. This function allows me to work a table of data rather than fitting a multidimensional polynomial type function. RegularizeData3D is an extremely useful tool.

I have been using this almost a week everyday. I work with a lot of tables of data. This function allows me to work a table of data rather than fitting a multidimensional polynomial type function. RegularizeData3D is an extremely useful tool.

I just started using this function today. I am amazed at how effective this function is at smoothing while maintaining the accuracy and integrity of the 3d surface. I definitely recommend use of this function. Well done!

joneauin

Hi I would love to try this software, my z values are stored in a matrix though, is it possible to use this matrix as input for RegularizeData3D?

Chad Greene

Chad Greene (view profile)

This function provides an excellent solution for fitting surfaces to irregularly-spaced data using a bicubic fit. Generating a 200,000 element surface from 65,000 GPS measurments takes about six seconds on my laptop.

A fine contribution to FEX.

Updates

1.1

The smoothing parameter now produces consistent behavior for input datasets with a very large or very small domain. (Previously the smoothing parameter was affected by the size of the input domain in x and y.)

MATLAB Release
MATLAB 8.3 (R2014a)
Acknowledgements

Inspired by: Surface Fitting using gridfit

Inspired: jasonnicholson/regularizeNd

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video