MATLAB Central Newsreader - Array of Images Running out of memory

Array of Images Running out of memory

Arron S — Tue, 06 Jan 2009 05:14:02 -0500

I am very very new to matlab and have written my first M-file. It is functional for a test set of 38 images but my data sets are in the range of 400. (144x512)

When I try to run it with a full data set I get an obvious memory error. I realize that I am running 32bit XP and thus don't have a large enough contiguous memory for my data. But I know that others are able to get around this and I would like to know how. This should be easy to answer since I have read a lot of posts where people have arrays of a good deal of images but they did not explain how they were doing it. I looked at a few hundred file exchange files and searched this forum before I decided to post this message. I did not find an answer that I understood. Any help or links to topics pointing me in the right direction would be very helpful.

My aim is to load about 400 images and run FRET analysis.
Below is my code (probably not optimal, and itis currently molded for my data with no input/error checks yet) I used ReadDicom4.m by Nipun Patel to teach myself some of the basics and hence some of the variables are of the same name.

Thank you in advance,
Arron

function a = imagestack(sequenceStartNo,sequenceEndNo, fileExtension)
tic

fileCount=0;
totalFiles = sequenceEndNo - sequenceStartNo;
a= zeros(144,512,totalFiles); % preallocation

j=1;

for i=sequenceStartNo:sequenceEndNo

    sequenceNo=num2str(i);
    filename = strcat(sequenceNo,fileExtension);
    varname = imread (filename);
    varname = mat2gray(varname);

    fileCount = fileCount + 1;

    a(:,:,j)= varname;

    j = j+1;
end

toc
display (fileCount);

Re: Array of Images Running out of memory

Nasser Abbasi — Tue, 06 Jan 2009 07:15:51 -0500

"Arron S" <aps@uwm.edu> wrote in message
news:gjupaq$jeu$1@fred.mathworks.com...
>I am very very new to matlab and have written my first M-file. It is
>functional for a test set of 38 images but my data sets are in the range of
>400. (144x512)
>

400*144*512*4 is only about 118 MBytes I do not see why your PC can't
manage this. Here is a small test on my Matlab

EDU>> clear all
EDU>> memory
Maximum possible array: 1120 MB (1.174e+009 bytes) *
Memory available for all arrays: 1529 MB (1.603e+009 bytes) **
Memory used by MATLAB: 245 MB (2.573e+008 bytes)
Physical Memory (RAM): 3327 MB (3.489e+009 bytes)

* Limited by contiguous virtual address space available.
** Limited by virtual address space available.

EDU>> A=zeros(144,512,400);

EDU>> memory
Maximum possible array: 895 MB (9.383e+008 bytes) *
Memory available for all arrays: 1304 MB (1.367e+009 bytes) **
Memory used by MATLAB: 470 MB (4.932e+008 bytes)
Physical Memory (RAM): 3327 MB (3.489e+009 bytes)

* Limited by contiguous virtual address space available.
** Limited by virtual address space available.
EDU>>

So, you see that memory used by Matlab increased from 245 to 470, or by 225
MB only. So, I am not sure why you are getting the memory error. How much
RAM does your PC have? Why not set a break point on error, and look at the
memory then? May be you are using memory somewhere else in the program?

--Nasser

Re: Array of Images Running out of memory

Pete — Tue, 06 Jan 2009 13:16:02 -0500

Hi,

I think it should be,
totalFiles = sequenceEndNo - sequenceStartNo + 1;

By underestimating the total number of files, the preallocated array isn't quite big enough and adding the last image results in a new, larger array being created - as far as I know the original array cannot be expanded, so about twice as much memory is needed (at least temporarily).

If you still have memory problems, you could consider using the single rather than double data type, which should halve the memory requirements. You would preallocate using
a = zeros(144, 512, totalFiles, 'single');

A 144x512x400 single array would be about 112.5 MB, while the double array would be 225 MB. You might not need the extra precision of using double.

Depending upon how the images are stored, you also might be able to skip the call to mat2gray.

All the best,

Pete

Re: Array of Images Running out of memory

Steven Lord — Tue, 06 Jan 2009 15:00:38 -0500

"Arron S" <aps@uwm.edu> wrote in message
news:gjupaq$jeu$1@fred.mathworks.com...
>I am very very new to matlab and have written my first M-file. It is
>functional for a test set of 38 images but my data sets are in the range of
>400. (144x512)
>
> When I try to run it with a full data set I get an obvious memory error.
> I realize that I am running 32bit XP and thus don't have a large enough
> contiguous memory for my data. But I know that others are able to get
> around this and I would like to know how. This should be easy to answer
> since I have read a lot of posts where people have arrays of a good deal
> of images but they did not explain how they were doing it. I looked at a
> few hundred file exchange files and searched this forum before I decided
> to post this message. I did not find an answer that I understood. Any
> help or links to topics pointing me in the right direction would be very
> helpful.
>
> My aim is to load about 400 images and run FRET analysis.
> Below is my code (probably not optimal, and itis currently molded for my
> data with no input/error checks yet) I used ReadDicom4.m by Nipun Patel to
> teach myself some of the basics and hence some of the variables are of the
> same name.

The first question I'd ask is: do you need to have all 38 (or 400) images
loaded into memory at the same time? Can you instead read in one image,
perform the necessary calculations, and then repeat for the remainder of
your images?

Pete's suggestion about using a different data type may help as well,
particularly if the images can be stored and processed as one of the integer
data types (like int8 or uint8, which would use one eighth the memory of
storing it as a double.) If all the images can be stored as int8's, for
instance, then preallocate with:

a = zeros(144, 512, totalFiles, 'int8');

--
Steve Lord
slord@mathworks.com

Re: Array of Images Running out of memory

Arron S — Tue, 06 Jan 2009 19:57:01 -0500

Thank you all for the help. I believe the initial problem was that I had fragmented memory from testing because it worked this morning after i restarted the computer with the large data set. But I also think that processing each image individually will also be a great help for future calculation and I appreciate that tip. When I get this running I will post again for tips to make it file exchange ready for future users who want to do FRET.
Thanks again,
Arron

"Arron S" <aps@uwm.edu> wrote in message <gjupaq$jeu$1@fred.mathworks.com>...
> I am very very new to matlab and have written my first M-file. It is functional for a test set of 38 images but my data sets are in the range of 400. (144x512)
>
> When I try to run it with a full data set I get an obvious memory error. I realize that I am running 32bit XP and thus don't have a large enough contiguous memory for my data. But I know that others are able to get around this and I would like to know how. This should be easy to answer since I have read a lot of posts where people have arrays of a good deal of images but they did not explain how they were doing it. I looked at a few hundred file exchange files and searched this forum before I decided to post this message. I did not find an answer that I understood. Any help or links to topics pointing me in the right direction would be very helpful.
>
> My aim is to load about 400 images and run FRET analysis.
> Below is my code (probably not optimal, and itis currently molded for my data with no input/error checks yet) I used ReadDicom4.m by Nipun Patel to teach myself some of the basics and hence some of the variables are of the same name.
>
> Thank you in advance,
> Arron
>
>
>
> function a = imagestack(sequenceStartNo,sequenceEndNo, fileExtension)
> tic
>
>
> fileCount=0;
> totalFiles = sequenceEndNo - sequenceStartNo;
> a= zeros(144,512,totalFiles); % preallocation
>
> j=1;
>
> for i=sequenceStartNo:sequenceEndNo
>
> sequenceNo=num2str(i);
> filename = strcat(sequenceNo,fileExtension);
> varname = imread (filename);
> varname = mat2gray(varname);
>
> fileCount = fileCount + 1;
>
> a(:,:,j)= varname;
>
> j = j+1;
> end
>
> toc
> display (fileCount);
>
>