Thread Subject:

matlab/mex function doesn't free memory

Hello all,

I'm running a mex function that calls for c code (compiled in the same file as the mex function itself).
when a simulation is done, it seems matlab doesn't release the memory it uses since i see a growing amount of memory being used by matlab (via task manager).

it reaches the limit of my computer and then either crashes or simply continues to run forever.

any suggestion on how to solve this problem is very appriciated.

Look for memory leak in the C/mex code.

Bruno

"Alex Berkovich" <alexber@tx.technion> wrote in message <gi3i3u$m5q$1@fred.mathworks.com>...
> Hello all,
>
> I'm running a mex function that calls for c code (compiled in the same file as the mex function itself).
> when a simulation is done, it seems matlab doesn't release the memory it uses since i see a growing amount of memory being used by matlab (via task manager).
>
> it reaches the limit of my computer and then either crashes or simply continues to run forever.
>
> any suggestion on how to solve this problem is very appriciated.
>

should I free memory / variables / do other actions, other that free to any variable that i did memory allocation to?

thanks a lot!

> should I free memory / variables / do other actions, other that free to any variable that i did memory allocation to?
>

Call free and use MxDestroyArray, and you should be fine. I believe the later is even not necessary because Matlab should automatically destroy the one use forget to do. I always do that manually though.

Bruno

On 14 Des, 19:09, "Alex Berkovich" <alex...@tx.technion> wrote:
> Hello all,
>
> I'm running a mex function that calls for c code (compiled in the same file as the mex function itself).
> when a simulation is done, it seems matlab doesn't release the memory it uses since i see a growing amount of memory being used by matlab (via task manager).
>
> it reaches the limit of my computer and then either crashes or simply continues to run forever.
>
> any suggestion on how to solve this problem is very appriciated.

Any data structures you allocate with malloc-type functions
(or with new in C++) should be freed by the appropriate
corresponding calls to free or delete before your MEX function
returns.

As for mxCreateXXXX, only use them for whatever you intend to
return back to the matlab workspace. I've never understood how
the matlab memory managment system works (it seems to be based
on some Java-type garbage collector) and I've always ended up
with segmentation faults when I used mxFree on local variables
inside the MEX function.

Rune

On Dec 14, 2:35=A0pm, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 14 Des, 19:09, "Alex Berkovich" <alex...@tx.technion> wrote:
>
> > Hello all,
>
> > I'm running a mex function that calls for c code (compiled in the same =
file as the mex function itself).
> > when a simulation is done, it seems matlab doesn't release the memory i=
t uses since i see a growing amount of memory being used by matlab (via tas=
k manager).
>
> > it reaches the limit of my computer and then either crashes or simply c=
ontinues to run forever.
>
> > any suggestion on how to solve this problem is very appriciated.
>
> Any data structures you allocate with malloc-type functions
> (or with new in C++) should be freed by the appropriate
> corresponding calls to free or delete before your MEX function
> returns.
>
> As for mxCreateXXXX, only use them for whatever you intend to
> return back to the matlab workspace. I've never understood how
> the matlab memory managment system works (it seems to be based
> on some Java-type garbage collector) and I've always ended up
> with segmentation faults when I used mxFree on local variables
> inside the MEX function.
>
> Rune

If you're using mxFree on variables created using mxCreate* functions
it'll always result in a memory leak. mxFree will only free the
mxArray header and not the data pointed to by the header. Use
mxDestroyArray to free variables allocated using mxCreate* and mxFree
for those allocated using mxCalloc, mxMalloc and mxRealloc. I only use
the MEX API functions to manage memory in C mex functions and it works
just fine.

However, in case of C++ files it's better to use new except for
mxArray variables as you mentioned, and then free these using delete
in the class destructor. The MEX API does do it's own garbage
collection whenever MATLAB terminates a MEX file (because of a call to
mexErrMsgIdAndTxt for instance) before the mexFunction stack is
unwound. In this case trying to free memory using mxDestroyArray or
mxFree in the destructor causes a segv (since the memory has already
been freed by the garbage collector).

Praetorian <ashish.sadanandan@gmail.com> wrote in message <c70f9832-2682-4e8b-a87a-06175c26c62a@b41g2000pra.googlegroups.com>...
> On Dec 14, 2:35=A0pm, Rune Allnor <all...@tele.ntnu.no> wrote:
> > On 14 Des, 19:09, "Alex Berkovich" <alex...@tx.technion> wrote:
> >
> > > Hello all,
> >
> > > I'm running a mex function that calls for c code (compiled in the same =
> file as the mex function itself).
> > > when a simulation is done, it seems matlab doesn't release the memory i=
> t uses since i see a growing amount of memory being used by matlab (via tas=
> k manager).
> >
> > > it reaches the limit of my computer and then either crashes or simply c=
> ontinues to run forever.
> >
> > > any suggestion on how to solve this problem is very appriciated.
> >
> > Any data structures you allocate with malloc-type functions
> > (or with new in C++) should be freed by the appropriate
> > corresponding calls to free or delete before your MEX function
> > returns.
> >
> > As for mxCreateXXXX, only use them for whatever you intend to
> > return back to the matlab workspace. I've never understood how
> > the matlab memory managment system works (it seems to be based
> > on some Java-type garbage collector) and I've always ended up
> > with segmentation faults when I used mxFree on local variables
> > inside the MEX function.
> >
> > Rune
>
> If you're using mxFree on variables created using mxCreate* functions
> it'll always result in a memory leak. mxFree will only free the
> mxArray header and not the data pointed to by the header. Use
> mxDestroyArray to free variables allocated using mxCreate* and mxFree
> for those allocated using mxCalloc, mxMalloc and mxRealloc. I only use
> the MEX API functions to manage memory in C mex functions and it works
> just fine.
>
> However, in case of C++ files it's better to use new except for
> mxArray variables as you mentioned, and then free these using delete
> in the class destructor. The MEX API does do it's own garbage
> collection whenever MATLAB terminates a MEX file (because of a call to
> mexErrMsgIdAndTxt for instance) before the mexFunction stack is
> unwound. In this case trying to free memory using mxDestroyArray or
> mxFree in the destructor causes a segv (since the memory has already
> been freed by the garbage collector).

I have a question close to that one..
You said that "The MEX API does do it's own garbage collection whenever MATLAB terminates a MEX file", fine, but what about a function called in a shared library (via calllib) that return a mxArray* and thus, alloc via mxCreate functions ? Is this mxArray destoyed in C++ workspace ?
Thanks..

"Alex Berkovich" <alexber@tx.technion> wrote in message <gi3i3u$m5q$1@fred.mathworks.com>...
> Hello all,
>
> I'm running a mex function that calls for c code (compiled in the same file as the mex function itself).
> when a simulation is done, it seems matlab doesn't release the memory it uses since i see a growing amount of memory being used by matlab (via task manager).
>
> it reaches the limit of my computer and then either crashes or simply continues to run forever.
>
> any suggestion on how to solve this problem is very appriciated.
>

Hi all,

It DOES seem that MatLab leaks memory when running mex files. Take a look at the following: Below is a c-mex file (trace2plane.c) that finds intersections between hyperplanes and polygonal-line traces. Run the test code (test_trace2plane.m, down at the bottom) several times and you'll discover MatLab is taking up more and more memory in the task manager. I've been trying to figure out the problem for almost a month now, any insigths or help of any kind will be much appreciated.

Gabriel



/* trace2plane.c */
#include "mex.h"
#include <math.h>

#define min(a, b) (((a) < (b)) ? (a) : (b))
#define max(a, b) (((a) > (b)) ? (a) : (b))
#define sqr(a) ((a)*(a))
#define abs(a) (((a) > (0)) ? (a) : (-a))
#define sgn(a) (((a) > (0)) ? (1) : (-1))

/* Gateway foutine. */
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
  
  /* Declare variables. */
  int i, j, k;
  double *center, *direction, *trace;
  double **position, **velocity, *location, *distance;
  int **member;
  double *pointer, dot, norm, delta;
  int length, dimension, total, product, index;
  int size = 0, count = 0, increment;
  char buffer[100];
  
  /* Check argument count. */
  if (nrhs < 3)
    mexErrMsgTxt("Not enough inputs.");
  if (nlhs > 5)
    mexErrMsgTxt("Too many outputs.");
  
  /* Check input class. */
  for (i = 0; i < nrhs; i++) {
    if (!(mxIsNumeric(prhs[i]))) {
      sprintf(buffer, "Input %d must be numeric.", i + 1);
      mexErrMsgTxt(buffer);
    }
    if (mxIsComplex(prhs[i])) {
      sprintf(buffer, "Input %d must be real.", i + 1);
      mexErrMsgTxt(buffer);
    }
    if (!(mxIsDouble(prhs[i]))) {
      sprintf(buffer, "Input %d must be of class double.", i + 1);
      mexErrMsgTxt(buffer);
    }
  }
  
  /* Check input size. */
  dimension = mxGetN(prhs[0]);
  for (i = 0; i < nrhs; i++) {
    if (mxGetNumberOfDimensions(prhs[i]) > 2) {
      sprintf(buffer, "Input %d must be a matrix.", i + 1);
      mexErrMsgTxt(buffer);
    }
    if (mxGetN(prhs[i]) != dimension) {
      sprintf(buffer, "Input %d must have %d columns.", i + 1, dimension);
      mexErrMsgTxt(buffer);
    }
  }
  total = mxGetM(prhs[0]);
  if (mxGetM(prhs[1]) != total) {
    sprintf(buffer, "Input 2 must have %d rows.", total);
    mexErrMsgTxt(buffer);
  }
  product = mxGetNumberOfElements(prhs[0]);
  
  /* Check for early return. */
  if (mxIsEmpty(prhs[0]) || mxIsEmpty(prhs[2])) {
    plhs[0] = mxCreateNumericMatrix(count, dimension, mxDOUBLE_CLASS, mxREAL);
    if (nlhs > 1) {
      plhs[1] = mxCreateNumericMatrix(count, dimension, mxDOUBLE_CLASS, mxREAL);
      if (nlhs > 2) {
        plhs[2] = mxCreateNumericMatrix(count, 2, mxDOUBLE_CLASS, mxREAL);
        if (nlhs > 3) {
          plhs[3] = mxCreateNumericMatrix(count, 1, mxDOUBLE_CLASS, mxREAL);
          if (nlhs > 4) {
            plhs[4] = mxCreateNumericMatrix(count, 1, mxDOUBLE_CLASS, mxREAL);
          }
        }
      }
    }
    return;
  }
  
  /* Allocate space. */
  increment = nrhs - 2;
  size = increment;
  position = mxMalloc(size*sizeof(double*));
  velocity = mxMalloc(size*sizeof(double*));
  member = mxMalloc(size*sizeof(int*));
  for (i = 0; i < size; i++) {
    position[i] = mxMalloc(dimension*sizeof(double));
    velocity[i] = mxMalloc(dimension*sizeof(double));
    member[i] = mxMalloc(2*sizeof(int));
  }
  location = mxMalloc(size*sizeof(double));
  distance = mxMalloc(size*sizeof(double));
  
  /* Get pointer. */
  center = mxGetPr(prhs[0]);
  direction = mxGetPr(prhs[1]);
  for (i = 0; i < total; i++) {
    
    /* Calculate projections. */
    for (j = 2; j < nrhs; j++) {
      trace = mxGetPr(prhs[j]);
      length = mxGetM(prhs[j]);
      for (k = 0; k < length - 1; k++) {
        dot = 0.0;
        delta = 0.0;
        pointer = trace + k;
        for (index = i; index < product; index += total, pointer += length) {
          dot += direction[index]*(center[index] - *pointer);
          delta += direction[index]*(*(pointer + 1) - *pointer);
        }
        if (delta != 0.0) {
          dot /= delta;
          if ((dot > 0.0) && (dot <= 1.0)) {
            
            /* Allocate more space if necessary. */
            if (count >= size) {
              size += increment;
              position = mxRealloc(position, size*sizeof(double*));
              velocity = mxRealloc(velocity, size*sizeof(double*));
              member = mxRealloc(member, size*sizeof(int*));
              for (index = size - increment; index < size; index++) {
                position[index] = mxMalloc(dimension*sizeof(double));
                velocity[index] = mxMalloc(dimension*sizeof(double));
                member[index] = mxMalloc(2*sizeof(int));
              }
              location = mxRealloc(location, size*sizeof(double));
              distance = mxRealloc(distance, size*sizeof(double));
            }
            
            /* Append. */
            norm = 0.0;
            pointer = trace + k;
            for (index = 0; index < dimension; index++, pointer += length) {
              delta = *(pointer + 1) - *pointer;
              position[count][index] = *pointer + dot*delta;
              velocity[count][index] = delta;
              delta = center[i + index*dimension] - position[count][index];
              norm += sqr(delta);
            }
            member[count][0] = i + 1;
            member[count][1] = j - 1;
            location[count] = dot + (double) (k + 1);
            distance[count] = sqrt(norm);
            count++;
            
          }
        }
      }
    }
    
    /* Trim. */
    for (i = size - count; k < size; k++) {
      mxFree(position[i]);
      mxFree(velocity[i]);
      mxFree(member[i]);
    }
    position = mxRealloc(position, count*sizeof(double*));
    velocity = mxRealloc(velocity, count*sizeof(double*));
    member = mxRealloc(member, count*sizeof(int*));
    location = mxRealloc(location, count*sizeof(double));
    distance = mxRealloc(distance, count*sizeof(double));
    
  }
  
  /* Fill in position. */
  plhs[0] = mxCreateNumericMatrix(count, dimension, mxDOUBLE_CLASS, mxREAL);
  pointer = mxGetPr(plhs[0]);
  for (j = 0; j < dimension; j++)
    for (i = 0; i < count; i++) {
      *pointer = position[i][j];
      pointer++;
    }
  if (nlhs > 1) {
    
    /* Fill in velocity. */
    plhs[1] = mxCreateNumericMatrix(count, dimension, mxDOUBLE_CLASS, mxREAL);
    pointer = mxGetPr(plhs[1]);
    for (j = 0; j < dimension; j++)
      for (i = 0; i < count; i++) {
        *pointer = velocity[i][j];
        pointer++;
      }
    if (nlhs > 2) {
      
      /* Fill in member. */
      plhs[2] = mxCreateNumericMatrix(count, 2, mxDOUBLE_CLASS, mxREAL);
      pointer = mxGetPr(plhs[2]);
      for (j = 0; j < 2; j++)
        for (i = 0; i < count; i++) {
          *pointer = (double) member[i][j];
          pointer++;
        }
      if (nlhs > 3) {
        
        /* Fill in location. */
        plhs[3] = mxCreateNumericMatrix(count, 1, mxDOUBLE_CLASS, mxREAL);
        pointer = mxGetPr(plhs[3]);
        for (i = 0; i < count; i++)
          pointer[i] = location[i];
        if (nlhs > 4) {
          
          /* Fill in distance. */
          plhs[4] = mxCreateNumericMatrix(count, 1, mxDOUBLE_CLASS, mxREAL);
          pointer = mxGetPr(plhs[4]);
          for (i = 0; i < count; i++)
            pointer[i] = distance[i];
          
        }
      }
    }
  }
  
  /* Free space. */
  for (i = 0; i < size; i++) {
    mxFree(position[i]);
    mxFree(velocity[i]);
    mxFree(member[i]);
  }
  mxFree(position);
  mxFree(velocity);
  mxFree(member);
  mxFree(location);
  mxFree(distance);
  
}





function test_trace2plane

%-------------------------------------------------------------------------%
% First case %
%-------------------------------------------------------------------------%

% Generate data.
length = 5;
dimension = 2;
center = zeros(1, dimension);
direction = randn(1, dimension);
trace = randn(length, dimension);
trace = cumsum(trace, 1);
trace = bsxfun(@minus, trace, mean(trace, 1));
trace = bsxfun(@times, trace, 1./std(trace, 1, 1));

% Project.
[position, velocity, member, location, distance] = ...
    trace2plane(center, direction, trace);

% Create line.
x = transpose(linspace(-1.0, 1.0, length));
[q, r] = qr(transpose(direction));
i = eq(r, 0.0);
x = bsxfun(@plus, x*q(i, :), center);

% Plot.
figure
set(gca, ...
    'Color', get(gcf, 'Color'), ...
    'XGrid', 'on', ...
    'YGrid', 'on', ...
    'DataAspectRatio', ones(1, 3), ...
    'NextPlot', 'add')
quiver(center(:, 1), center(:, 2), ...
    direction(:, 1), direction(:, 2), ...
    'Color', 'r', ...
    'Marker', '.', ...
    'AutoScale', 'off')
plot(x(:, 1), x(:, 2), ...
    'Color', 'r', ...
    'LineStyle', '--')
plot(trace(:, 1), trace(:, 2),...
    'Color', 'b', ...
    'Marker', '.')
plot(transpose(cat(2, ...
        center(ones(size(position, 1), 1), 1), ...
        position(:, 1))), ...
    transpose(cat(2, ...
        center(ones(size(position, 1), 1), 2), ...
        position(:, 2))), ...
    'Color', 'y', ...
    'Marker', 'none', ...
    'LineStyle', '-')
for i = 1:length
    text(trace(i, 1), trace(i, 2), ...
        sprintf('%d', i), ...
        'HorizontalAlignment', 'Left', ...
        'VerticalAlignment', 'Bottom')
end
for i = 1:size(position, 1)
    text(position(i, 1), position(i, 2), ...
        sprintf('%4.4f', location(i)), ...
        'Color', 'g', ...
        'HorizontalAlignment', 'Right', ...
        'VerticalAlignment', 'Top')
    text(0.5*(center(1) + position(i, 1)), ...
        0.5*(center(2) + position(i, 2)), ...
        sprintf('%4.4f', distance(i)), ...
        'Color', 'y', ...
        'HorizontalAlignment', 'Right', ...
        'VerticalAlignment', 'Top')
end
plot(position(:, 1), position(:, 2), ...
    'Color', 'g', ...
    'Marker', '.', ...
    'LineStyle', 'none')

%-------------------------------------------------------------------------%
% Second case %
%-------------------------------------------------------------------------%

% Generate data.
length = 5;
dimension = 3;
center = zeros(1, dimension);
direction = randn(1, dimension);
trace = randn(length, dimension);
trace = cumsum(trace, 1);
trace = bsxfun(@minus, trace, mean(trace, 1));
trace = bsxfun(@times, trace, 1./std(trace, 1, 1));

% Project.
[position, velocity, member, location, distance] = ...
    trace2plane(center, direction, trace);

% Create plane.
x = cell(1, 2);
[x{:}] = meshgrid(transpose(linspace(-1.0, 1.0, length)));
for i = 1:2
    x{i} = x{i}(:);
end
x = cat(2, x{:});
[q, r] = qr(transpose(direction));
i = eq(r, 0.0);
x = bsxfun(@plus, x*q(i, :), center);
x = mat2cell(x, length^2, ones(1, 3));
for i = 1:3
    x{i} = reshape(x{i}, length, length);
end

% Plot.
figure
set(gca, ...
    'Color', get(gcf, 'Color'), ...
    'XGrid', 'on', ...
    'YGrid', 'on', ...
    'ZGrid', 'on', ...
    'DataAspectRatio', ones(1, 3), ...
    'NextPlot', 'add')
quiver3(center(:, 1), center(:, 2), center(:, 3), ...
    direction(:, 1), direction(:, 2), direction(:, 3), ...
    'Color', 'r', ...
    'Marker', '.', ...
    'AutoScale', 'off')
surf(x{1}, x{2}, x{3}, ...
    'EdgeColor', 'r', ...
    'FaceColor', 'none')
plot3(trace(:, 1), trace(:, 2), trace(:, 3), ...
    'Color', 'b', ...
    'Marker', '.')
plot3(transpose(cat(2, ...
        center(ones(size(position, 1), 1), 1), ...
        position(:, 1))), ...
    transpose(cat(2, ...
        center(ones(size(position, 1), 1), 2), ...
        position(:, 2))), ...
    transpose(cat(2, ...
        center(ones(size(position, 1), 1), 3), ...
        position(:, 3))), ...
    'Color', 'y', ...
    'Marker', 'none', ...
    'LineStyle', '-')
for i = 1:length
    text(trace(i, 1), trace(i, 2), trace(i, 3), ...
        sprintf('%d', i), ...
        'HorizontalAlignment', 'Left', ...
        'VerticalAlignment', 'Bottom')
end
for i = 1:size(position, 1)
    text(position(i, 1), position(i, 2), position(i, 3), ...
        sprintf('%4.4f', location(i)), ...
        'Color', 'g', ...
        'HorizontalAlignment', 'Right', ...
        'VerticalAlignment', 'Top')
    text(0.5*(center(1) + position(i, 1)), ...
        0.5*(center(2) + position(i, 2)), ...
        0.5*(center(3) + position(i, 3)), ...
        sprintf('%4.4f', distance(i)), ...
        'Color', 'y', ...
        'HorizontalAlignment', 'Right', ...
        'VerticalAlignment', 'Top')
end
plot3(position(:, 1), position(:, 2), position(:, 3), ...
    'Color', 'g', ...
    'Marker', '.', ...
    'LineStyle', 'none')

return

"gabo " <pelusacorazon@hotmail.com> wrote in message <grgne9$1ng$1@fred.mathworks.com>...
> "Alex Berkovich" <alexber@tx.technion> wrote in message <gi3i3u$m5q$1@fred.mathworks.com>...
> > Hello all,
> >
> > I'm running a mex function that calls for c code (compiled in the same file as the mex function itself).
> > when a simulation is done, it seems matlab doesn't release the memory it uses since i see a growing amount of memory being used by matlab (via task manager).
> >
> > it reaches the limit of my computer and then either crashes or simply continues to run forever.
> >
> > any suggestion on how to solve this problem is very appriciated.
> >
>
> Hi all,
>
> It DOES seem that MatLab leaks memory when running mex files. Take a look at the following: Below is a c-mex file (trace2plane.c) that finds intersections between hyperplanes and polygonal-line traces. Run the test code (test_trace2plane.m, down at the bottom) several times and you'll discover MatLab is taking up more and more memory in the task manager. I've been trying to figure out the problem for almost a month now, any insigths or help of any kind will be much appreciated.
>
> Gabriel
>
-------------------------------------------------------------------------------------

Here are my results:

>> mex trace2plane.c
>> feature memstats
                                            1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
                                            1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
                                            1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
                                            1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
                                            1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
                                            1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
                                            1505 MB (5e1d1000)
>> test_trace2plane
>> feature memstats
                                            1505 MB (5e1d1000)
>>

Other than a bunch of figures building up, I don't see any large increases in memory usage from call to call. So I can't reproduce your problem. WinXP with MSVC 8.0. How many times do you have to run it to see a problem? How much memory are you losing with each call?

James Tursa

On Apr 8, 12:48=A0am, "James Tursa" <aclassyguywithakno...@hotmail.com>
wrote:
> "gabo " <pelusacora...@hotmail.com> wrote in message <grgne9$1n...@fred.m=
athworks.com>...
> > "Alex Berkovich" <alex...@tx.technion> wrote in message <gi3i3u$m5...@f=
red.mathworks.com>...
> > > Hello all,
>
> > > I'm running a mex function that calls for c code (compiled in the sam=
e file as the mex function itself).
> > > when a simulation is done, it seems matlab doesn't release the memory=
 it uses since i see a growing amount of memory being used by matlab (via t=
ask manager).
>
> > > it reaches the limit of my computer and then either crashes or simply=
 continues to run forever.
>
> > > any suggestion on how to solve this problem is very appriciated.
>
> > Hi all,
>
> > It DOES seem that MatLab leaks memory when running mex files. Take a lo=
ok at the following: Below is a c-mex file (trace2plane.c) that finds inter=
sections between hyperplanes and polygonal-line traces. Run the test code (=
test_trace2plane.m, down at the bottom) several times and you'll discover M=
atLab is taking up more and more memory in the task manager. I've been tryi=
ng to figure out the problem for almost a month now, any insigths or help o=
f any kind will be much appreciated.
>
> > Gabriel
>
> -------------------------------------------------------------------------=
------------
>
> Here are my results:
>
> >> mex trace2plane.c
> >> feature memstats
>
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 1505 MB (5e1d1000)>> test_trace2plane
> >> feature memstats
>
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 1505 MB (5e1d1000)>> test_trace2plane
> >> feature memstats
>
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 1505 MB (5e1d1000)>> test_trace2plane
> >> feature memstats
>
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 1505 MB (5e1d1000)>> test_trace2plane
> >> feature memstats
>
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 1505 MB (5e1d1000)>> test_trace2plane
> >> feature memstats
>
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 1505 MB (5e1d1000)>> test_trace2plane
> >> feature memstats
>
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 1505 MB (5e1d1000)>> test_trace2plane
> >> feature memstats
>
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 1505 MB (5e1d1000)
>
>
>
> Other than a bunch of figures building up, I don't see any large increase=
s in memory usage from call to call. So I can't reproduce your problem. Win=
XP with MSVC 8.0. How many times do you have to run it to see a problem? Ho=
w much memory are you losing with each call?
>
> James Tursa

James,
I was hoping you'd see this thread. Take a look at the post from Ronan
on the 4th of February. It was directed to me but I didn't see it
until last night. Do you have any experience with calling a DLL that
uses the mxCreate* functions to allocate memory? And does the MATLAB
memory manager garbage collect that memory assuming that the DLL is
loaded and used by a mex file?

Regards,
Ashish.

Praetorian <ashish.sadanandan@gmail.com> wrote in message <251c8005-d072-419f-ad9a-28680bdb1707@g19g2000yql.googlegroups.com>...
>
> James,
> I was hoping you'd see this thread. Take a look at the post from Ronan
> on the 4th of February. It was directed to me but I didn't see it
> until last night. Do you have any experience with calling a DLL that
> uses the mxCreate* functions to allocate memory? And does the MATLAB
> memory manager garbage collect that memory assuming that the DLL is
> loaded and used by a mex file?
>
> Regards,
> Ashish.

Well, I didn't have any experience with this when I first read this post, but now I do. The answer is no, MATLAB does *not* do garbage collection on mx___ allocated memory in a loaded dll. This is different from mex functions, where garbage collection *is* done. Example files are below. Commands I used to show the memory leak are:

mex mexmem.c
mexmem
feature memstats

This will show the garbage collection being done for mex functions. Inside the function, 100MB is allocated but not explicitly deallocated. The feature memstats done inside the mex routine show the allocation. But upon returning to MATLAB and doing a feature memstats reveals that the memory is in fact freed.

Now build the dllmem dll (I used a mex command with a custom mexopts.bat file to build the dll library) and then do this:

loadlibrary('dllmem.mexw32','dllmem.h')
feature memstats
calllib('dllmem','memallocate')
feature memstats
unloadlibrary('dllmem')
feature memstats

You can see that the 100MB has been leaked. Garbage collection did *not* take place when the dll library was unloaded.

Bottom line: Free the memory manually, no safety net here.

James Tursa

--------------------------------------------------------------------------------------------------
mexmem.c

#include "mex.h"

double *dp = NULL;

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    mexEvalString("feature memstats");
    if( dp == NULL ) {
        dp = mxMalloc(13107200*sizeof(double)); // allocate 100MB
    }
    mexEvalString("feature memstats");
}

-------------------------------------------------------------------------------------------------------
dllmem.c

#include "mex.h"

double *dp = NULL;

void memallocate(void)
{
    if( dp == NULL ) {
        dp = mxMalloc(13107200*sizeof(double)); // allocate 100MB
    }
}

void memdeallocate(void)
{
    if( dp ) {
        mxFree(dp);
    }
    dp = NULL;
}

-----------------------------------------------------------------------------------------------------
dllmem.h

void memallocate(void);
void memdeallocate(void);

----------------------------------------------------------------------------------------------------
dllmem.def

EXPORTS
memallocate
memdeallocate

On Apr 8, 9:20=A0pm, "James Tursa" <aclassyguywithakno...@hotmail.com>
wrote:
> Praetorian <ashish.sadanan...@gmail.com> wrote in message <251c8005-d072-=
419f-ad9a-28680bdb1...@g19g2000yql.googlegroups.com>...
>
> > James,
> > I was hoping you'd see this thread. Take a look at the post from Ronan
> > on the 4th of February. It was directed to me but I didn't see it
> > until last night. Do you have any experience with calling a DLL that
> > uses the mxCreate* functions to allocate memory? And does the MATLAB
> > memory manager garbage collect that memory assuming that the DLL is
> > loaded and used by a mex file?
>
> > Regards,
> > Ashish.
>
> Well, I didn't have any experience with this when I first read this post,=
 but now I do. =A0The answer is no, MATLAB does *not* do garbage collection=
 on mx___ allocated memory in a loaded dll. This is different from mex func=
tions, where garbage collection *is* done. =A0Example files are below. =A0C=
ommands I used to show the memory leak are:
>
> mex mexmem.c
> mexmem
> feature memstats
>
> This will show the garbage collection being done for mex functions. Insid=
e the function, 100MB is allocated but not explicitly deallocated. The feat=
ure memstats done inside the mex routine show the allocation. But upon retu=
rning to MATLAB and doing a feature memstats reveals that the memory is in =
fact freed.
>
> Now build the dllmem dll (I used a mex command with a custom mexopts.bat =
file to build the dll library) and then do this:
>
> loadlibrary('dllmem.mexw32','dllmem.h')
> feature memstats
> calllib('dllmem','memallocate')
> feature memstats
> unloadlibrary('dllmem')
> feature memstats
>
> You can see that the 100MB has been leaked. Garbage collection did *not* =
take place when the dll library was unloaded.
>
> Bottom line: Free the memory manually, no safety net here.
>
> James Tursa
>
> -------------------------------------------------------------------------=
-------------------------
> mexmem.c
>
> #include "mex.h"
>
> double *dp =3D NULL;
>
> void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs=
[])
> {
> =A0 =A0 mexEvalString("feature memstats");
> =A0 =A0 if( dp =3D=3D NULL ) {
> =A0 =A0 =A0 =A0 dp =3D mxMalloc(13107200*sizeof(double)); =A0// allocate =
100MB
> =A0 =A0 }
> =A0 =A0 mexEvalString("feature memstats");
>
> }
>
> -------------------------------------------------------------------------=
------------------------------
> dllmem.c
>
> #include "mex.h"
>
> double *dp =3D NULL;
>
> void memallocate(void)
> {
> =A0 =A0 if( dp =3D=3D NULL ) {
> =A0 =A0 =A0 =A0 dp =3D mxMalloc(13107200*sizeof(double)); =A0// allocate =
100MB
> =A0 =A0 }
>
> }
>
> void memdeallocate(void)
> {
> =A0 =A0 if( dp ) {
> =A0 =A0 =A0 =A0 mxFree(dp);
> =A0 =A0 }
> =A0 =A0 dp =3D NULL;
>
> }
>
> -------------------------------------------------------------------------=
----------------------------
> dllmem.h
>
> void memallocate(void);
> void memdeallocate(void);
>
> -------------------------------------------------------------------------=
---------------------------
> dllmem.def
>
> EXPORTS
> memallocate
> memdeallocate


Thanks for coding up that example! So I went ahead and did things a
little differently; instead of using CALLLIB from the command line I
loaded the DLL using the Windows API LoadLibrary function from within
mexmem. And what do you know, MATLAB garbage collects the memory in
that case. The only other difference was that I created a Visual
Studio project to compile my DLL instead of modifying mexopts.bat but
I can't think of any project setting that would cause this change in
behavior. Here's my modified mexmem.c


---------------------------------------------------------------------------=
---------------------------
mexmem.c


#include "mex.h"
#include "windows.h"

double *dp =3D NULL;

/* mexmem - allocates memory within mexFunction, does not
free it
 * mexmem( false ) - same as first call
 * mexmem( true ) - calls dllmem.dll which allocates memory, does
not free it
 * mexmem( true, ... ) - calls dllmem.dll which deallocates memory
 */

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray
*prhs[])
{
  mexEvalString("feature memstats");
  if( ( nrhs =3D=3D 0 ) || ( !mxIsLogicalScalarTrue(prhs[0]) ) ) {
    if( dp =3D=3D NULL ) {
      dp =3D mxMalloc(100*1024*1024*sizeof(double)); //100MB * sizeof
(double)
      mexPrintf( "Memory allocated at: %px\n", dp );
    } else {
      mexPrintf( "Memory not allocated\n" );
    }
  } else {
    HMODULE hLib =3D LoadLibrary( "dllmem.dll" );

    mexPrintf( "Loading dllmem\n" );

    if( hLib =3D=3D NULL ) {
      mexPrintf( "Failed to load dllmem" );
    } else {
      FARPROC pAlloc =3D GetProcAddress( hLib, "memallocate" );
      FARPROC pDealloc =3D GetProcAddress( hLib, "memdeallocate" );

      if( pAlloc =3D=3D NULL ) {
        mexPrintf( "Failed to get memallocate address" );
      } else if( pDealloc =3D=3D NULL ) {
        mexPrintf( "Failed to get memdeallocate address" );
      }

      if( nrhs > 1 ) {
        (pDealloc)();
      } else {
        (pAlloc)();
      }

      if( FreeLibrary( hLib ) =3D=3D 0 ) {
        mexPrintf( "Failed to unload dllmem" );
      }
    }
  }

  mexEvalString("feature memstats");
}
---------------------------------------------------------------------------=
---------

The calls I made at the command line were:

%To test allocation within mexmem
mexmem
feature memstats
[x x] =3D inmem;
any(ismember('mexmem', x))

%To test allocation within dllmem
mexmem( true )
feature memstats
[x x] =3D inmem;
any(ismember('mexmem', x))

I added the INMEM check because I was always under the impression that
MATLAB didn't do any mex garbage collection until the mex file was
cleared from memory. But these tests prove otherwise! Also, I tried
commenting out the FreeLibrary call to see what would happen if the
DLL remained in memory but that gave the same results.

Any thoughts on why CALLLIB doesn't work the same way?

Regards,
Ashish.

On Apr 9, 11:23=A0am, Praetorian <ashish.sadanan...@gmail.com> wrote:
> On Apr 8, 9:20=A0pm, "James Tursa" <aclassyguywithakno...@hotmail.com>
> wrote:
>
>
>
> > Praetorian <ashish.sadanan...@gmail.com> wrote in message <251c8005-d07=
2-419f-ad9a-28680bdb1...@g19g2000yql.googlegroups.com>...
>
> > > James,
> > > I was hoping you'd see this thread. Take a look at the post from Rona=
n
> > > on the 4th of February. It was directed to me but I didn't see it
> > > until last night. Do you have any experience with calling a DLL that
> > > uses the mxCreate* functions to allocate memory? And does the MATLAB
> > > memory manager garbage collect that memory assuming that the DLL is
> > > loaded and used by a mex file?
>
> > > Regards,
> > > Ashish.
>
> > Well, I didn't have any experience with this when I first read this pos=
t, but now I do. =A0The answer is no, MATLAB does *not* do garbage collecti=
on on mx___ allocated memory in a loaded dll. This is different from mex fu=
nctions, where garbage collection *is* done. =A0Example files are below. =
=A0Commands I used to show the memory leak are:
>
> > mex mexmem.c
> > mexmem
> > feature memstats
>
> > This will show the garbage collection being done for mex functions. Ins=
ide the function, 100MB is allocated but not explicitly deallocated. The fe=
ature memstats done inside the mex routine show the allocation. But upon re=
turning to MATLAB and doing a feature memstats reveals that the memory is i=
n fact freed.
>
> > Now build the dllmem dll (I used a mex command with a custom mexopts.ba=
t file to build the dll library) and then do this:
>
> > loadlibrary('dllmem.mexw32','dllmem.h')
> > feature memstats
> > calllib('dllmem','memallocate')
> > feature memstats
> > unloadlibrary('dllmem')
> > feature memstats
>
> > You can see that the 100MB has been leaked. Garbage collection did *not=
* take place when the dll library was unloaded.
>
> > Bottom line: Free the memory manually, no safety net here.
>
> > James Tursa
>
> > -----------------------------------------------------------------------=
---------------------------
> > mexmem.c
>
> > #include "mex.h"
>
> > double *dp =3D NULL;
>
> > void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *pr=
hs[])
> > {
> > =A0 =A0 mexEvalString("feature memstats");
> > =A0 =A0 if( dp =3D=3D NULL ) {
> > =A0 =A0 =A0 =A0 dp =3D mxMalloc(13107200*sizeof(double)); =A0// allocat=
e 100MB
> > =A0 =A0 }
> > =A0 =A0 mexEvalString("feature memstats");
>
> > }
>
> > -----------------------------------------------------------------------=
--------------------------------
> > dllmem.c
>
> > #include "mex.h"
>
> > double *dp =3D NULL;
>
> > void memallocate(void)
> > {
> > =A0 =A0 if( dp =3D=3D NULL ) {
> > =A0 =A0 =A0 =A0 dp =3D mxMalloc(13107200*sizeof(double)); =A0// allocat=
e 100MB
> > =A0 =A0 }
>
> > }
>
> > void memdeallocate(void)
> > {
> > =A0 =A0 if( dp ) {
> > =A0 =A0 =A0 =A0 mxFree(dp);
> > =A0 =A0 }
> > =A0 =A0 dp =3D NULL;
>
> > }
>
> > -----------------------------------------------------------------------=
------------------------------
> > dllmem.h
>
> > void memallocate(void);
> > void memdeallocate(void);
>
> > -----------------------------------------------------------------------=
-----------------------------
> > dllmem.def
>
> > EXPORTS
> > memallocate
> > memdeallocate
>
> Thanks for coding up that example! So I went ahead and did things a
> little differently; instead of using CALLLIB from the command line I
> loaded the DLL using the Windows API LoadLibrary function from within
> mexmem. And what do you know, MATLAB garbage collects the memory in
> that case. The only other difference was that I created a Visual
> Studio project to compile my DLL instead of modifying mexopts.bat but
> I can't think of any project setting that would cause this change in
> behavior. Here's my modified mexmem.c
>
> -------------------------------------------------------------------------=
-----------------------------
> mexmem.c
>
> #include "mex.h"
> #include "windows.h"
>
> double *dp =3D NULL;
>
> /* mexmem =A0 =A0 =A0 =A0 =A0 - allocates memory within mexFunction, does=
 not
> free it
> =A0* mexmem( false ) =A0- same as first call
> =A0* mexmem( true ) =A0 - calls dllmem.dll which allocates memory, does
> not free it
> =A0* mexmem( true, ... ) - calls dllmem.dll which deallocates memory
> =A0*/
>
> void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray
> *prhs[])
> {
> =A0 mexEvalString("feature memstats");
> =A0 if( ( nrhs =3D=3D 0 =A0) || ( !mxIsLogicalScalarTrue(prhs[0]) ) ) {
> =A0 =A0 if( dp =3D=3D NULL ) {
> =A0 =A0 =A0 dp =3D mxMalloc(100*1024*1024*sizeof(double)); =A0//100MB * s=
izeof
> (double)
> =A0 =A0 =A0 mexPrintf( "Memory allocated at: %px\n", dp );
> =A0 =A0 } else {
> =A0 =A0 =A0 mexPrintf( "Memory not allocated\n" );
> =A0 =A0 }
> =A0 } else {
> =A0 =A0 HMODULE hLib =A0=3D LoadLibrary( "dllmem.dll" );
>
> =A0 =A0 mexPrintf( "Loading dllmem\n" );
>
> =A0 =A0 if( hLib =3D=3D NULL ) {
> =A0 =A0 =A0 mexPrintf( "Failed to load dllmem" );
> =A0 =A0 } else {
> =A0 =A0 =A0 FARPROC pAlloc =A0 =A0=3D GetProcAddress( hLib, "memallocate"=
 );
> =A0 =A0 =A0 FARPROC pDealloc =A0=3D GetProcAddress( hLib, "memdeallocate"=
 );
>
> =A0 =A0 =A0 if( pAlloc =3D=3D NULL ) {
> =A0 =A0 =A0 =A0 mexPrintf( "Failed to get memallocate address" );
> =A0 =A0 =A0 } else if( pDealloc =3D=3D NULL ) {
> =A0 =A0 =A0 =A0 mexPrintf( "Failed to get memdeallocate address" );
> =A0 =A0 =A0 }
>
> =A0 =A0 =A0 if( nrhs > 1 ) {
> =A0 =A0 =A0 =A0 (pDealloc)();
> =A0 =A0 =A0 } else {
> =A0 =A0 =A0 =A0 (pAlloc)();
> =A0 =A0 =A0 }
>
> =A0 =A0 =A0 if( FreeLibrary( hLib ) =3D=3D 0 ) {
> =A0 =A0 =A0 =A0 mexPrintf( "Failed to unload dllmem" );
> =A0 =A0 =A0 }
> =A0 =A0 }
> =A0 }
>
> =A0 mexEvalString("feature memstats");}
>
> -------------------------------------------------------------------------=
-----------
>
> The calls I made at the command line were:
>
> %To test allocation within mexmem
> mexmem
> feature memstats
> [x x] =3D inmem;
> any(ismember('mexmem', x))
>
> %To test allocation within dllmem
> mexmem( true )
> feature memstats
> [x x] =3D inmem;
> any(ismember('mexmem', x))
>
> I added the INMEM check because I was always under the impression that
> MATLAB didn't do any mex garbage collection until the mex file was
> cleared from memory. But these tests prove otherwise! Also, I tried
> commenting out the FreeLibrary call to see what would happen if the
> DLL remained in memory but that gave the same results.
>
> Any thoughts on why CALLLIB doesn't work the same way?
>
> Regards,
> Ashish.

Oh and I forgot to mention that any attempt to have dllmem.dll
deallocate the memory results in a segv, which is to be expected.

Praetorian <ashish.sadanandan@gmail.com> wrote in message <a018942f-5682-44ca-82bf-ea96eb58c8dd@a23g2000vbl.googlegroups.com>...
>
> Thanks for coding up that example! So I went ahead and did things a
> little differently; instead of using CALLLIB from the command line I
> loaded the DLL using the Windows API LoadLibrary function from within
> mexmem. And what do you know, MATLAB garbage collects the memory in
> that case. The only other difference was that I created a Visual
> Studio project to compile my DLL instead of modifying mexopts.bat but
> I can't think of any project setting that would cause this change in
> behavior. Here's my modified mexmem.c
>
> (snip)
>
> I added the INMEM check because I was always under the impression that
> MATLAB didn't do any mex garbage collection until the mex file was
> cleared from memory. But these tests prove otherwise! Also, I tried
> commenting out the FreeLibrary call to see what would happen if the
> DLL remained in memory but that gave the same results.
>
> Any thoughts on why CALLLIB doesn't work the same way?
>
> Regards,
> Ashish.

Ashish,

I haven't had a chance to try out your example yet ... maybe this weekend. But I did modify some other dll code that I have that is a basic memory sharing example. Called it using Windows API functions from within a mex routine. The dll has functions for allocating memory, copying data to the memory, displaying the memory, and deallocating the memory. The dll works great using malloc and free, but when I switch to mxMalloc and mxFree the dll craps out with a seg fault. This doesn't always happen right away, but it always happens eventually. So I suspect that there is a fundamental restriction here that you can't use MATLAB API functions inside a shared library dll because it messes up the MATLAB memory manager in some way ... they are probably walking on each other. You apparently experienced seg faults as well when deallocating with one of your examples. So the garbage collection issue
is probably a moot point for shared library dll's since using any of the MATLAB API functions that allocate or deallocate memory will screw up the MATLAB memory manager and cause a seg fault.

James Tursa

On Apr 10, 2:24=A0am, "James Tursa" <aclassyguywithakno...@hotmail.com>
wrote:
> Praetorian <ashish.sadanan...@gmail.com> wrote in message <a018942f-5682-=
44ca-82bf-ea96eb58c...@a23g2000vbl.googlegroups.com>...
>
> > Thanks for coding up that example! So I went ahead and did things a
> > little differently; instead of using CALLLIB from the command line I
> > loaded the DLL using the Windows API LoadLibrary function from within
> > mexmem. And what do you know, MATLAB garbage collects the memory in
> > that case. The only other difference was that I created a Visual
> > Studio project to compile my DLL instead of modifying mexopts.bat but
> > I can't think of any project setting that would cause this change in
> > behavior. Here's my modified mexmem.c
>
> > (snip)
>
> > I added the INMEM check because I was always under the impression that
> > MATLAB didn't do any mex garbage collection until the mex file was
> > cleared from memory. But these tests prove otherwise! Also, I tried
> > commenting out the FreeLibrary call to see what would happen if the
> > DLL remained in memory but that gave the same results.
>
> > Any thoughts on why CALLLIB doesn't work the same way?
>
> > Regards,
> > Ashish.
>
> Ashish,
>
> I haven't had a chance to try out your example yet ... maybe this weekend=
. But I did modify some other dll code that I have that is a basic memory s=
haring example. Called it using Windows API functions from within a mex rou=
tine. The dll has functions for allocating memory, copying data to the memo=
ry, displaying the memory, and deallocating the memory. The dll works great=
 using malloc and free, but when I switch to mxMalloc and mxFree the dll cr=
aps out with a seg fault. This doesn't always happen right away, but it alw=
ays happens eventually. So I suspect that there is a fundamental restrictio=
n here that you can't use MATLAB API functions inside a shared library dll =
because it messes up the MATLAB memory manager in some way ... they are pro=
bably walking on each other. You apparently experienced seg faults as well =
when deallocating with one of your examples. So the garbage collection issu=
e
> is probably a moot point for shared library dll's since using any of the =
MATLAB API functions that allocate or deallocate memory will screw up the M=
ATLAB memory manager and cause a seg fault.
>
> James Tursa

I'm not so sure you can't use the mxMalloc and mxFree from within a
shared library. I modified the code so that I was calling the DLL's
allocate function followed immediately by the deallocate function; I
then called the mex file from the command line 1000 times and didn't
get any seg faults. The reason I got one yesterday was because I
allocated memory using the DLL in one call to the mex file and then
tried to deallocate it during the next call. But my tests from
yesterday show that the Matlab memory manager does garbage collection
on the memory allocated by the DLL when the calling mex file exits. So
the memory allocated in the first call had already been deallocated
(by the memory manager) and I tried to explicitly deallocate it in the
next call resulting in the segv.

Ashish.

Praetorian <ashish.sadanandan@gmail.com> wrote in message <b70044a2-8b1c-4fbf-b475-122750fdd38f@a5g2000pre.googlegroups.com>...
>
> I'm not so sure you can't use the mxMalloc and mxFree from within a
> shared library. I modified the code so that I was calling the DLL's
> allocate function followed immediately by the deallocate function; I
> then called the mex file from the command line 1000 times and didn't
> get any seg faults. The reason I got one yesterday was because I
> allocated memory using the DLL in one call to the mex file and then
> tried to deallocate it during the next call. But my tests from
> yesterday show that the Matlab memory manager does garbage collection
> on the memory allocated by the DLL when the calling mex file exits. So
> the memory allocated in the first call had already been deallocated
> (by the memory manager) and I tried to explicitly deallocate it in the
> next call resulting in the segv.
>
> Ashish.

Ah, got it. Thanks! I put in a mexMakeMemoryPersistent call in the dll and now everything works ok. That puts the burdon of clearing the memory on the dll programmer, but that's really as it should be. I tested my example using both API calls from within a mex and from the MATLAB prompt using loadlibrary and calllib. Works in both cases, no memory leak and no memory corruption. The dll detects when it is being detached from MATLAB, so the programmer can put code in the dll to automatically clear the memory in that event.

James Tursa

"Ronan " <ronan.trepos@toulouse.inra.fr> wrote in message <gmc94h$rl0$1@fred.mathworks.com>...
> Praetorian <ashish.sadanandan@gmail.com> wrote in message <c70f9832-2682-4e8b-a87a-06175c26c62a@b41g2000pra.googlegroups.com>...
> > On Dec 14, 2:35=A0pm, Rune Allnor <all...@tele.ntnu.no> wrote:
> > > On 14 Des, 19:09, "Alex Berkovich" <alex...@tx.technion> wrote:
> > >
> > > > Hello all,
> > >
> > > > I'm running a mex function that calls for c code (compiled in the same =
> > file as the mex function itself).
> > > > when a simulation is done, it seems matlab doesn't release the memory i=
> > t uses since i see a growing amount of memory being used by matlab (via tas=
> > k manager).
> > >
> > > > it reaches the limit of my computer and then either crashes or simply c=
> > ontinues to run forever.
> > >
> > > > any suggestion on how to solve this problem is very appriciated.
> > >
> > > Any data structures you allocate with malloc-type functions
> > > (or with new in C++) should be freed by the appropriate
> > > corresponding calls to free or delete before your MEX function
> > > returns.
> > >
> > > As for mxCreateXXXX, only use them for whatever you intend to
> > > return back to the matlab workspace. I've never understood how
> > > the matlab memory managment system works (it seems to be based
> > > on some Java-type garbage collector) and I've always ended up
> > > with segmentation faults when I used mxFree on local variables
> > > inside the MEX function.
> > >
> > > Rune
> >
> > If you're using mxFree on variables created using mxCreate* functions
> > it'll always result in a memory leak. mxFree will only free the
> > mxArray header and not the data pointed to by the header. Use
> > mxDestroyArray to free variables allocated using mxCreate* and mxFree
> > for those allocated using mxCalloc, mxMalloc and mxRealloc. I only use
> > the MEX API functions to manage memory in C mex functions and it works
> > just fine.
> >
> > However, in case of C++ files it's better to use new except for
> > mxArray variables as you mentioned, and then free these using delete
> > in the class destructor. The MEX API does do it's own garbage
> > collection whenever MATLAB terminates a MEX file (because of a call to
> > mexErrMsgIdAndTxt for instance) before the mexFunction stack is
> > unwound. In this case trying to free memory using mxDestroyArray or
> > mxFree in the destructor causes a segv (since the memory has already
> > been freed by the garbage collector).
>
> I have a question close to that one..
> You said that "The MEX API does do it's own garbage collection whenever MATLAB terminates a MEX file", fine, but what about a function called in a shared library (via calllib) that return a mxArray* and thus, alloc via mxCreate functions ? Is this mxArray destoyed in C++ workspace ?
> Thanks..
Hi Ronan,
Currently I meet exactly the same problem. I wrote a function in C which returns mxArray* to Matlab, and this function is called in a shared library as well, and it will be called continuously. To free memory which occupied by return mxArray I wrote a function ( using mxDestroyArray), however, Matlab crashed, and seems it cannot find the location of returned mxArray. Is that becaues Matlab has it own garbage collection and after every call of function, it frees the memory occupied by returned mxArray?
Have you figured this issue out? I will appreciate a lot if you can give me any hint:)

"Xueming " <emilyhexueming@hotmail.com> wrote in message <k9l55g$j9c$1@newscl01ah.mathworks.com>...

> Currently I meet exactly the same problem. I wrote a function in C which returns mxArray* to Matlab, and this function is called in a shared library as well, and it will be called continuously. To free memory which occupied by return mxArray I wrote a function ( using mxDestroyArray), however, Matlab crashed, and seems it cannot find the location of returned mxArray. Is that becaues Matlab has it own garbage collection and after every call of function, it frees the memory occupied by returned mxArray?
> Have you figured this issue out? I will appreciate a lot if you can give me any hint:)

In general, you should *not* be using mxDestroyArray on anything that is being returned to MATLAB. That will invalidate the data in the mxArray and cause MATLAB to crash.

James Tursa

"James Tursa" wrote in message <k9lhsc$b0h$1@newscl01ah.mathworks.com>...
> "Xueming " <emilyhexueming@hotmail.com> wrote in message <k9l55g$j9c$1@newscl01ah.mathworks.com>...
>
> > Currently I meet exactly the same problem. I wrote a function in C which returns mxArray* to Matlab, and this function is called in a shared library as well, and it will be called continuously. To free memory which occupied by return mxArray I wrote a function ( using mxDestroyArray), however, Matlab crashed, and seems it cannot find the location of returned mxArray. Is that becaues Matlab has it own garbage collection and after every call of function, it frees the memory occupied by returned mxArray?
> > Have you figured this issue out? I will appreciate a lot if you can give me any hint:)
>
> In general, you should *not* be using mxDestroyArray on anything that is being returned to MATLAB. That will invalidate the data in the mxArray and cause MATLAB to crash.
>
> James Tursa

Hi James,
Now I have two issues,
1) I create a C function, which will be called by Matlab, and when everytime it's called, it create a mxArray and return this pointer to Matlab. With called more and more times, it creates more and more mxArray. I guess the memory must be significantly occupied. I'm thinking if we can mxDestory a mxArray (by writing another function) as soon as it's returned to Matlab, so that we only allocate memory for current mxArray.

2)How about inside C code, if declaring a global mxArray*, how to make this global pointer point to a mxArray which is returned by a C function.
I tried to declare a global mxArray*, and set it point to a mxArray and returned to Matlab. This global mxArray* is used in other functions. But it seems in other functions, the global mxArray pointer could not be found where it points to. I got some strange result. Could anyone tell me the reason. Here is a simple code.
if run this code, calllib('mxArrayTest', 'create'), calllib('mxArrayTest','test'); the result of retruing mxCell1 and mxCell_g are completely different. The free_mxelem(mxCell_g) works pretty well just following mxCell_g=mxCell1, however, outside the "create" function, the free_mxelem() shows pretty strange result. I assume this is because after C returns mxArray to Matlab, memory of mxArray will be free automatically.

here is my code:
/////////////////mxArrayTest.c
#include "mex.h" /* only needed because of mexFunction below and mexPrintf */
#include <math.h>
#include "mxArrayTest.h"
#include "string.h"



/*******************function declare***********************************/
#define mxNULL mxCreateDoubleMatrix(0,0,mxREAL)
mxArray *mxCell_g;

/*type check*/

void type_check(void)
{
int num;
int i;
mxCell_g=mxNULL;
num=mxGetN(mxCell_g);

if(mxIsEmpty(mxCell_g))
{
mexPrintf("Empty");
}
else
{
for(i=0;i<num;i++)
{
mexPrintf("type is: %s",mxGetClassName(mxCell_g));
}
}
}

/********************get element************************************/
 void getelement(int pos,mxArray* mxCell_g)
{
if(mxIsEmpty(mxCell_g))
{
mexPrintf("No element!\n");
}

else if(mxIsCell(mxGetCell(mxCell_g,pos)))
{
mexPrintf("This element is a cell!\n ");
}

else
{
mexPrintf("This elem is other type!\n");
}

}
/********************************************************/
 int free_mxelem(mxArray* mxelem)

{
if(mxIsEmpty(mxelem))
{
mexWarnMsgTxt("Empty Element!\n");
return 0;
}
else if(mxIsChar(mxelem)||mxIsDouble(mxelem))
{

mxDestroyArray(mxelem);
mexPrintf("Single Successful!\n");
return 1;
}
// else if(mxIsCell(mxelem))
else if(mxIsCell(mxelem))
{

int n;//number of element
int i;//counter
n=mxGetN(mxelem);
mexPrintf("Free Cell!\n");
for(i=0;i<n;i++)
{

free_mxelem(mxGetCell(mxelem,i));

}
mxDestroyArray(mxelem);
mexPrintf("Cell Successful!\n");
return 1;
}
return 0;

}



/**************test******************************/


DLL_EXPORT mxArray* create(void)
{
mxArray *mxDouble;
mxArray *mxCell;
mxArray *mxCell1;
mxArray *mxString;

double *a;
char *str;


mxDouble=mxCreateDoubleMatrix(1,2,mxREAL);
a=mxGetPr(mxDouble);
a[0]=1;
a[1]=2;

str=(char*)malloc(sizeof(char)*20);
strcpy(str,"mxArray Test");
mxString=mxCreateString(str);


mxCell=mxCreateCellMatrix(1,2);
mxSetCell(mxCell,0,mxDouble);
mxSetCell(mxCell,1,mxString);

mxCell1=mxCreateCellMatrix(1,2);
mxSetCell(mxCell1,0,mxCell);
mxSetCell(mxCell1,1,mxString);

mxCell_g=mxCell1;
// free_mxelem(mxCell_g);
mexPrintf("is this empty? %d\nClass: %s\n",mxIsEmpty(mxCell_g),mxGetClassName(mxCell_g));
// return mxCell_g;
return mxCell1;
}


DLL_EXPORT void test(void)
{
int free_mxelem(mxArray* mxelem);
mexPrintf("is this empty? %d\n%s\n",mxIsEmpty(mxCell_g),mxGetClassName(mxCell_g));
free_mxelem(mxCell_g);

}

////////////////////////mxArrayTest.h
#ifndef _MXARRAYTEST_H_
#define _MXARRAYTEST_H_
#define DLL_EXPORT __declspec(dllexport)

#ifdef __cplusplus
extern "C" {
#endif

#include "mex.h"


//free_element
//DLL_EXPORT void free_elem(void);

/*test*/
DLL_EXPORT void test(void);
DLL_EXPORT mxArray* create(void);
DLL_EXPORT void type_check(void);

#ifdef __cplusplus
}
#endif

#endif

/////////////////end

If this piece of code and this message are messy for, I do apologize, since I'm a really beginer with this.

Emily

"Xueming " <emilyhexueming@hotmail.com> wrote in message <k9n94g$gtu$1@newscl01ah.mathworks.com>...
>
> Hi James,
> Now I have two issues,
> 1) I create a C function, which will be called by Matlab, and when everytime it's called, it create a mxArray and return this pointer to Matlab. With called more and more times, it creates more and more mxArray. I guess the memory must be significantly occupied. I'm thinking if we can mxDestory a mxArray (by writing another function) as soon as it's returned to Matlab, so that we only allocate memory for current mxArray.

I'm not really sure what you are doing with (1) above, so can't comment. If you show the associated code then I could comment.

> 2)How about inside C code, if declaring a global mxArray*, how to make this global pointer point to a mxArray which is returned by a C function.
> I tried to declare a global mxArray*, and set it point to a mxArray and returned to Matlab. This global mxArray* is used in other functions. But it seems in other functions, the global mxArray pointer could not be found where it points to. I got some strange result. Could anyone tell me the reason. Here is a simple code.
> if run this code, calllib('mxArrayTest', 'create'), calllib('mxArrayTest','test'); the result of retruing mxCell1 and mxCell_g are completely different. The free_mxelem(mxCell_g) works pretty well just following mxCell_g=mxCell1, however, outside the "create" function, the free_mxelem() shows pretty strange result. I assume this is because after C returns mxArray to Matlab, memory of mxArray will be free automatically.

For (2) above, it appears from your code that you have a fundamental misunderstanding of how garbage collection works with API function calls. All of the official API functions for creating mxArray variables (i.e., the mxCreateEtc functions) put the address of the created mxArray on the garbage collection list, which I will call the Variable Array List (VAL) for lack of a better term. When the mex function or calllib function returns control back to the caller (i.e., MATLAB), everything on the VAL is destroyed. It appears that that is what is going on in your code. To make a mxArray persist and not be garbage collected between calls, you must remove its address from the VAL prior to the function exiting and returning control back to the caller. The function for this is mexMakeArrayPersistent. E.g., in this code snippet:

> mxCell1=mxCreateCellMatrix(1,2);
> mxSetCell(mxCell1,0,mxCell);
> mxSetCell(mxCell1,1,mxString);
>
> mxCell_g=mxCell1;

Your mxCell_g is your global mxArray that you want to persist between calls. So you need to follow up the above code with this line which will remove mxCell_g from the VAL:

       mexMakeArrayPersistent(mxCell_g);

CAUTION: Once you make the above call, mxCell_g is permanently removed from the VAL and there is no way to get it back on the VAL that I know of. So you risk a permanent memory leak if you don't have adequate code in place to destroy this mxArray. E.g., a function within the library to destroy this mxArray (so you can destroy it manually), and code in place that detects when the library is being detached from MATLAB (e.g., unloaded from memory) so that it can destroy the mxArray prior to being unloaded. Etc.

(btw, There is a similar function, mexMakeMemoryPersistent, for raw memory allocated with mxMalloc and friends that removes it from garbage collection. In that case, there *is* an undocumented function to get it back on the garbage collection list.)

Another suggestion I will make is to keep your global pointers in a valid state at all times. E.g., do this declaration instead of the one you have:

mxArray *mxCell_g = NULL;

That way it starts in a testable state indicating there is no variable present. When you create a variable, always test mxCell_g first, and if it is not NULL then destroy it before replacing it with another allocated mxArray. And whenever you destroy the variable in mxCell_g, set mxCell_g = NULL. Etc. That way you can always test the value of mxCell_g anywhere in your code to determine its state and if there is a variable present.

Also, I would note that the following code:

> str=(char*)malloc(sizeof(char)*20);
> strcpy(str,"mxArray Test");
> mxString=mxCreateString(str);

can be replaced with this simpler line:

  mxString=mxCreateString("mxArray Test");

The code you have is cumbersome and leaks the memory behind the str pointer.

James Tursa

"James Tursa" wrote in message <k9ob7p$onu$1@newscl01ah.mathworks.com>...
> "Xueming " <emilyhexueming@hotmail.com> wrote in message <k9n94g$gtu$1@newscl01ah.mathworks.com>...
> >
> > Hi James,
> > Now I have two issues,
> > 1) I create a C function, which will be called by Matlab, and when everytime it's called, it create a mxArray and return this pointer to Matlab. With called more and more times, it creates more and more mxArray. I guess the memory must be significantly occupied. I'm thinking if we can mxDestory a mxArray (by writing another function) as soon as it's returned to Matlab, so that we only allocate memory for current mxArray.
>
> I'm not really sure what you are doing with (1) above, so can't comment. If you show the associated code then I could comment.
>
> > 2)How about inside C code, if declaring a global mxArray*, how to make this global pointer point to a mxArray which is returned by a C function.
> > I tried to declare a global mxArray*, and set it point to a mxArray and returned to Matlab. This global mxArray* is used in other functions. But it seems in other functions, the global mxArray pointer could not be found where it points to. I got some strange result. Could anyone tell me the reason. Here is a simple code.
> > if run this code, calllib('mxArrayTest', 'create'), calllib('mxArrayTest','test'); the result of retruing mxCell1 and mxCell_g are completely different. The free_mxelem(mxCell_g) works pretty well just following mxCell_g=mxCell1, however, outside the "create" function, the free_mxelem() shows pretty strange result. I assume this is because after C returns mxArray to Matlab, memory of mxArray will be free automatically.
>
> For (2) above, it appears from your code that you have a fundamental misunderstanding of how garbage collection works with API function calls. All of the official API functions for creating mxArray variables (i.e., the mxCreateEtc functions) put the address of the created mxArray on the garbage collection list, which I will call the Variable Array List (VAL) for lack of a better term. When the mex function or calllib function returns control back to the caller (i.e., MATLAB), everything on the VAL is destroyed. It appears that that is what is going on in your code. To make a mxArray persist and not be garbage collected between calls, you must remove its address from the VAL prior to the function exiting and returning control back to the caller. The function for this is mexMakeArrayPersistent. E.g., in this code snippet:
>
> > mxCell1=mxCreateCellMatrix(1,2);
> > mxSetCell(mxCell1,0,mxCell);
> > mxSetCell(mxCell1,1,mxString);
> >
> > mxCell_g=mxCell1;
>
> Your mxCell_g is your global mxArray that you want to persist between calls. So you need to follow up the above code with this line which will remove mxCell_g from the VAL:
>
> mexMakeArrayPersistent(mxCell_g);
>
> CAUTION: Once you make the above call, mxCell_g is permanently removed from the VAL and there is no way to get it back on the VAL that I know of. So you risk a permanent memory leak if you don't have adequate code in place to destroy this mxArray. E.g., a function within the library to destroy this mxArray (so you can destroy it manually), and code in place that detects when the library is being detached from MATLAB (e.g., unloaded from memory) so that it can destroy the mxArray prior to being unloaded. Etc.
>
> (btw, There is a similar function, mexMakeMemoryPersistent, for raw memory allocated with mxMalloc and friends that removes it from garbage collection. In that case, there *is* an undocumented function to get it back on the garbage collection list.)
>
> Another suggestion I will make is to keep your global pointers in a valid state at all times. E.g., do this declaration instead of the one you have:
>
> mxArray *mxCell_g = NULL;
>
> That way it starts in a testable state indicating there is no variable present. When you create a variable, always test mxCell_g first, and if it is not NULL then destroy it before replacing it with another allocated mxArray. And whenever you destroy the variable in mxCell_g, set mxCell_g = NULL. Etc. That way you can always test the value of mxCell_g anywhere in your code to determine its state and if there is a variable present.
>
> Also, I would note that the following code:
>
> > str=(char*)malloc(sizeof(char)*20);
> > strcpy(str,"mxArray Test");
> > mxString=mxCreateString(str);
>
> can be replaced with this simpler line:
>
> mxString=mxCreateString("mxArray Test");
>
> The code you have is cumbersome and leaks the memory behind the str pointer.
>
> James Tursa
James,hank you very much for pointing out my misunderstanding of garbage collection mechanism, and my careless about pointer initialization. I found the memory management issues on Mathwork website, and gonna take a look very carefully.
Btw, I tried to put mexMakeArrayPersistent(mxCell_g);
following
mxSetCell(mxCell1,1,mxString);
mxCell_g=mxCell1;
but it did work. I was wondering since there is a cell in mxCell1, should I keep it from being destroyed as well?

Xueming

"Xueming " <emilyhexueming@hotmail.com> wrote in message <k9sg64$2c9$1@newscl01ah.mathworks.com>...
>
> James,hank you very much for pointing out my misunderstanding of garbage collection mechanism, and my careless about pointer initialization. I found the memory management issues on Mathwork website, and gonna take a look very carefully.
> Btw, I tried to put mexMakeArrayPersistent(mxCell_g);
> following
> mxSetCell(mxCell1,1,mxString);
> mxCell_g=mxCell1;
> but it did work. I was wondering since there is a cell in mxCell1, should I keep it from being destroyed as well?

What do you mean "... but it did not work"? Can you show your current code and what errors/problems you are currently encountering? As far as mxCell1 being a cell, that makes no difference ... mexMakeArrayPersistent works regardless of the type of variable argument.

James Tursa

"James Tursa" wrote in message <k9t4q4$ien$1@newscl01ah.mathworks.com>...
> "Xueming " <emilyhexueming@hotmail.com> wrote in message <k9sg64$2c9$1@newscl01ah.mathworks.com>...
> >
> > James,hank you very much for pointing out my misunderstanding of garbage collection mechanism, and my careless about pointer initialization. I found the memory management issues on Mathwork website, and gonna take a look very carefully.
> > Btw, I tried to put mexMakeArrayPersistent(mxCell_g);
> > following
> > mxSetCell(mxCell1,1,mxString);
> > mxCell_g=mxCell1;
> > but it did work. I was wondering since there is a cell in mxCell1, should I keep it from being destroyed as well?
>
> What do you mean "... but it did not work"? Can you show your current code and what errors/problems you are currently encountering? As far as mxCell1 being a cell, that makes no difference ... mexMakeArrayPersistent works regardless of the type of variable argument.
>
> James Tursa

Hi James,
my code was no big difference for the previous on, I just add mexMakeArrayPersistent(mxCell_g); and the code is like below:
{
        ...
mxCell1=mxCreateCellMatrix(1,2);
mxSetCell(mxCell1,0,mxCell);
mxSetCell(mxCell1,1,mxString);
mxCell_g=mxCell1;

mxCell_g=mxCell1;
mexMakeArrayPersistent(mxCell1);
return mxCell_g;
}

The result turned out as the same as without mexMakeArrayPersistent(mxCell1).

Emily