Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Memory leak writing NetCDF4 files but not NetCDF3 files

Subject: Memory leak writing NetCDF4 files but not NetCDF3 files

From: Jeff

Date: 3 May, 2013 03:14:10

Message: 1 of 3

One of my scripts used to write many large NetCDF3 files (80-200 MB each). For storage reasons, I changed this script to write to NetCDF4 files with compression:
Old: ncid=netcdf.create([outputfolder fname_nc], 'CLOBBER');
New: ncid=netcdf.create([outputfolder fname_nc], 'NETCDF4');
Unfortunately, after the new script runs, MATLAB doesn't free the memory used during the time the script runs. For example:

NetCDF3 - Before function runs:
 memory; gui_1('RAXPOL-20130417-225616.dat'); memory
Maximum possible array: 11806 MB (1.238e+010 bytes) *
Memory available for all arrays: 11806 MB (1.238e+010 bytes) *
Memory used by MATLAB: 1730 MB (1.815e+009 bytes)
Physical Memory (RAM): 16278 MB (1.707e+010 bytes)

NetCDF3 - After function runs:
Maximum possible array: 11359 MB (1.191e+010 bytes) *
Memory available for all arrays: 11359 MB (1.191e+010 bytes) *
Memory used by MATLAB: 1740 MB (1.825e+009 bytes)
Physical Memory (RAM): 16278 MB (1.707e+010 bytes)

NetCDF4 - Before function m-file runs:
>> memory; gui_1('RAXPOL-20130417-225616.dat'); memory
Maximum possible array: 12936 MB (1.356e+010 bytes) *
Memory available for all arrays: 12936 MB (1.356e+010 bytes) *
Memory used by MATLAB: 701 MB (7.355e+008 bytes)
Physical Memory (RAM): 16278 MB (1.707e+010 bytes)

NetCDF4 - After function m-file runs:
Maximum possible array: 11757 MB (1.233e+010 bytes) *
Memory available for all arrays: 11757 MB (1.233e+010 bytes) *
Memory used by MATLAB: 1716 MB (1.799e+009 bytes)
Physical Memory (RAM): 16278 MB (1.707e+010 bytes)

This is very repeatable and doesn't depend upon the particular files created. If I write to a NetCDF4 file, the memory usage grows and grows until Windows complains about low memory usage and Matlab crashes. It doesn't matter if I run the following:

close all
clear all
clear java
clear classes
clear functions
clear mex
pack

The memory used by MATLAB isn't freed, and I eventually need to restart Matlab to clear the memory. This only happens when writing NetCDF4 files.

I'm using Matlab 2011a on 64-bit Windows 7 with 16 GB of RAM. I preallocate all arrays, though that shouldn't matter since the memory should be cleared after the function "ends".

Any help? I don't want to have to restart Matlab so darn frequently just to free memory that should be freed automatically!

Subject: Memory leak writing NetCDF4 files but not NetCDF3 files

From: Ed Yu

Date: 24 May, 2013 18:40:10

Message: 2 of 3

Jeff,

It may have to do with the netcdf library implementation of the version 4 file handling... Maybe upgrading the library will help?

Good Luck!
Ed.

Subject: Memory leak writing NetCDF4 files but not NetCDF3 files

From: Ed Yu

Date: 24 May, 2013 18:47:11

Message: 3 of 3

Nevermind Jeff... I thought you were using the java library... I guess you'll have to talk to matlab technical support for this... Ed.

"Ed Yu" wrote in message <knoc69$1b4$1@newscl01ah.mathworks.com>...
> Jeff,
>
> It may have to do with the netcdf library implementation of the version 4 file handling... Maybe upgrading the library will help?
>
> Good Luck!
> Ed.

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us