Thread Subject:

Heap space error

I am running a heavy piece of code that after some time running its giving me an out of memory error. I tried several adjustments. However I just want to confirm something. Is it true that after I clear a variable (for example which stores an 800x800 image), that memory becomes reserved for variables smaller than the original variable size?

I think that my problem is that even though I'm freeing memory (by using clear), i'm running out of memory because of the lack of free contiguous memory. Any thoughts?

"Aron Sceidt" <sigarru87@gmail.com> wrote in message <j0k6o9$d8b$1@newscl01ah.mathworks.com>...
> I am running a heavy piece of code that after some time running its giving me an out of memory error. I tried several adjustments. However I just want to confirm something. Is it true that after I clear a variable (for example which stores an 800x800 image), that memory becomes reserved for variables smaller than the original variable size?
>
> I think that my problem is that even though I'm freeing memory (by using clear), i'm running out of memory because of the lack of free contiguous memory. Any thoughts?

answer any1?

"Aron Sceidt" <sigarru87@gmail.com> wrote in message <j0lonl$njk$1@newscl01ah.mathworks.com>...
> "Aron Sceidt" <sigarru87@gmail.com> wrote in message <j0k6o9$d8b$1@newscl01ah.mathworks.com>...
> > I am running a heavy piece of code that after some time running its giving me an out of memory error. I tried several adjustments. However I just want to confirm something. Is it true that after I clear a variable (for example which stores an 800x800 image), that memory becomes reserved for variables smaller than the original variable size?
> >
> > I think that my problem is that even though I'm freeing memory (by using clear), i'm running out of memory because of the lack of free contiguous memory. Any thoughts?
>
> answer any1?

Hi Aron,

a simple idea:

Maximum possible array: 700 MB (7.343e+008 bytes) *
Memory available for all arrays: 1437 MB (1.506e+009 bytes) **
Memory used by MATLAB: 352 MB (3.691e+008 bytes)
Physical Memory (RAM): 3511 MB (3.682e+009 bytes)

* Limited by contiguous virtual address space available.
** Limited by virtual address space available.
>> k = zeros(50000,1000);
>> memory
Maximum possible array: 319 MB (3.343e+008 bytes) *
Memory available for all arrays: 1055 MB (1.106e+009 bytes) **
Memory used by MATLAB: 734 MB (7.693e+008 bytes)
Physical Memory (RAM): 3511 MB (3.682e+009 bytes)

* Limited by contiguous virtual address space available.
** Limited by virtual address space available.
>> clear k
>> memory
Maximum possible array: 700 MB (7.343e+008 bytes) *
Memory available for all arrays: 1437 MB (1.506e+009 bytes) **
Memory used by MATLAB: 352 MB (3.693e+008 bytes)
Physical Memory (RAM): 3511 MB (3.682e+009 bytes)

does not look like your observation holds in the simple case of this command line test
have you watched the memory behaviour in debugging mode?

best
kinor

I just did and you're right.. doesn't show like its shrinking... Something very strange is happening as I keep getting a java heap space error after some time...

"kinor" wrote in message <j0lt7v$3lg$1@newscl01ah.mathworks.com>...
> "Aron Sceidt" <sigarru87@gmail.com> wrote in message <j0lonl$njk$1@newscl01ah.mathworks.com>...
> > "Aron Sceidt" <sigarru87@gmail.com> wrote in message <j0k6o9$d8b$1@newscl01ah.mathworks.com>...
> > > I am running a heavy piece of code that after some time running its giving me an out of memory error. I tried several adjustments. However I just want to confirm something. Is it true that after I clear a variable (for example which stores an 800x800 image), that memory becomes reserved for variables smaller than the original variable size?
> > >
> > > I think that my problem is that even though I'm freeing memory (by using clear), i'm running out of memory because of the lack of free contiguous memory. Any thoughts?
> >
> > answer any1?
>
> Hi Aron,
>
> a simple idea:
>
> Maximum possible array: 700 MB (7.343e+008 bytes) *
> Memory available for all arrays: 1437 MB (1.506e+009 bytes) **
> Memory used by MATLAB: 352 MB (3.691e+008 bytes)
> Physical Memory (RAM): 3511 MB (3.682e+009 bytes)
>
> * Limited by contiguous virtual address space available.
> ** Limited by virtual address space available.
> >> k = zeros(50000,1000);
> >> memory
> Maximum possible array: 319 MB (3.343e+008 bytes) *
> Memory available for all arrays: 1055 MB (1.106e+009 bytes) **
> Memory used by MATLAB: 734 MB (7.693e+008 bytes)
> Physical Memory (RAM): 3511 MB (3.682e+009 bytes)
>
> * Limited by contiguous virtual address space available.
> ** Limited by virtual address space available.
> >> clear k
> >> memory
> Maximum possible array: 700 MB (7.343e+008 bytes) *
> Memory available for all arrays: 1437 MB (1.506e+009 bytes) **
> Memory used by MATLAB: 352 MB (3.693e+008 bytes)
> Physical Memory (RAM): 3511 MB (3.682e+009 bytes)
>
> does not look like your observation holds in the simple case of this command line test
> have you watched the memory behaviour in debugging mode?
>
> best
> kinor

"Aron Sceidt" <sigarru87@gmail.com> wrote in message <j0m9li$ig$1@newscl01ah.mathworks.com>...
> I just did and you're right.. doesn't show like its shrinking... Something very strange is happening as I keep getting a java heap space error after some time...


I'm by no means an expert on memory management, but I'm pretty sure java heap space memory errors are completely unrelated to other memory management issues, since only graphics objects (along with all the desktop GUI elements) use the java heap space. I've hit these errors before while doing very memory-light calculations but generating dozens of plots from the same data. I'm not really sure what changes behind the scenes, but it seemed to me that different versions of Matlab were much more prone to these issues than others. It was never a problem for me up to release 2006 or so, became a little annoying in 2007 and then a major issue for me in R2008 (to the point that I downgraded back to 2007), and now using R2010 is seems to be back under control (my java heap space is a 256MB, and I only hit heap space errors on rare occasion now).

What version of Matlab are you running, and what is your java heap size? In more recent versions, you can increase the heap size under the Preferences menu. In older versions (R2009 and previous, if I remember correctly), you need to create a java.opts file (http://blogs.mathworks.com/desktop/2010/04/26/controlling-the-java-heap-size/).

I've found that if you turn off a lot of the little helper elements in the GUI (like mlint integrated warnings in the Editor), then that will also help reduce the java memory issues.

-Kelly