Out of memory. Type HELP MEMORY for your options.

3 views (last 30 days)
Hi, Please find below code.It gives Out of memory. HELP MEMORY for your options.Error in ndgrid (line 66) x = reshape(x(:,ones(1,prod(s))),[length(x) s]); . Please help me resolving this.
Thanks, Sita
clc; clear; tic p = haltonset(1) %'Skip',1e3,'Leap',1e2);
i1=1; j1=10;
i2=1; j2=10;
i3=1; j3=10;
i4=1; j4=10;
i5=1; j5=10;
i6=1; j6=10;
i7=1; j7=10;
i8=1; j8=10;
i9=1; j9=10;
i10=1; j10=10;
x1s(:,:)=p(i1:j1,:); x2s(:,:)=p(i2:j2,:); x3s(:,:)=p(i3:j3,:); x4s(:,:)=p(i4:j4,:); x5s(:,:)=p(i5:j5,:); x6s(:,:)=p(i6:j6,:); x7s(:,:)=p(i7:j7,:); x8s(:,:)=p(i8:j8,:); x9s(:,:)=p(i9:j9,:); x10s(:,:)=p(i10:j10,:);
[aa bb cc dd ee ff gg hh ii jj]=ndgrid(x1s,x2s,x3s,x4s,x5s,x6s,x7s,x8s,x9s,x10s);
overnested = arrayfun(@(x1,x2,x3,x4,x5,x6,x7,x8,x9,x10)[x1,x2,x3,x4,x5,x6,x7,x8,x9,x10],aa,bb,cc,dd,ee,ff,gg,hh,ii,jj,'un',0);
celldisp(overnested)
x = permute([aa(:), bb(:),cc(:),dd(:),ee(:),ff(:),gg(:), hh(:),ii(:),jj(:)],[ 11 10 9 8 7 6 5 4 3 2 1]);%replaced below code
x1temp=x(1,1,1,1,1,1,1,1,1,1,:)
x2temp=x(1,1,1,1,1,1,1,1,1,2,:)
x3temp=x(1,1,1,1,1,1,1,1,1,3,:)
x4temp=x(1,1,1,1,1,1,1,1,1,4,:)
x5temp=x(1,1,1,1,1,1,1,1,1,5,:)
x6temp=x(1,1,1,1,1,1,1,1,1,6,:)
x7temp=x(1,1,1,1,1,1,1,1,1,7,:)
x8temp=x(1,1,1,1,1,1,1,1,1,8,:)
x9temp=x(1,1,1,1,1,1,1,1,1,9,:)
x10temp=x(1,1,1,1,1,1,1,1,1,10,:)
  6 Comments
José-Luis
José-Luis on 17 May 2013
Edited: José-Luis on 17 May 2013
The usefulness of using a 10 dimensional ndgrid has to be questioned. Storing a double requires 64 bits or 8 bytes of memory. To give you an idea, assuming all dimensions are of equal length (all you x's have the same size), here are the storage requirements for 10-dimensional arrays:
  • size 2: 2^10 * 8 = 8192 bytes -> 8kB
  • size 5: 78125000 bytes -> 78MB
  • size 10: 8e10 bytes -> 80GB (out of range for a desktop nowadays)
  • size 20: 8.19e13 bytes -> 81TB (way out of theoretical range for 64bit computers)
I strongly suggest you revise your approach. Creating an ndgrid might not only be unnecessary, it is also probably a bad idea.
sita
sita on 17 May 2013
Hi, clear; clc; p = haltonset(1)
x1s=net(p,3) x2s=net(p,4) x3s=net(p,5) [a b c ]=ndgrid(x1s,x2s,x3s); overnested = arrayfun(@(x1,x2,x3)[x1 x2 x3],a,b,c,'un',0); celldisp(overnested) x = permute([a(:), b(:),c(:)],[4 3 2 1])
x1t=x(1,1,1,:) x2t=x(1,1,2,:) x3t=x(1,1,3,:) in above code x1s,x2s,x3s are my input vectors.I am trying to take all possible combinations of all the elements of 3 vectors.X is my vector with all possible combinations and i stored all three variables in x1t,x2t, x3t. After this i calculate my function where my inputs are x1t,x2t,x3t.To overcome from this memory problem, If i could store each row at a time(i.e. x1t,x2t,x3t) and calculate my function and clear that variable and repeat the same. Please help to fix this problem. Thanks, Sita

Sign in to comment.

Accepted Answer

José-Luis
José-Luis on 17 May 2013
Edited: José-Luis on 17 May 2013
All possible permutations in a loop:
for ii = 1:numel(x1)
for jj = 1:numel(x2)
for kk = 1:numel(x3)
your_input = [x1(ii) x2(jj) x3(kk)]
end
end
end
This could help with your out of memory error. Depending on the sizes of the vector it might take a long time. Please accept an answer if it helps you.
  4 Comments
sita
sita on 21 May 2013
Edited: sita on 21 May 2013
for i = 1:numel(x3s) x3=x3s(i) for j = 1:numel(x2s) x2=x2s(j) for k = 1:numel(x1s)
x1=x1s(k)
yal =fc10(x1,x2,x3)
your_input= [x1 x2 x3]
end
end
end hi, I edited my code and it is working.Though nested loops way solves memory problem it is very slow .If i want to do the same thing for ten variables i.e x1 ...x10. nested loops is very difficult.it takes very very long time.Please suggest me if there is any other method.
If i find a way of using permute instead of nested for loops with out memory problem,it helps to solve my problem.Please suggest me some method.
x = permute([aa(:), bb(:),cc(:),dd(:),ee(:),ff(:),gg(:), hh(:),ii(:),jj(:)],[ 11 10 9 8 7 6 5 4 3 2 1]);
José-Luis
José-Luis on 21 May 2013
You are trying to perform 10^10 operations. Depending on the complexity of your function, that is bound to take some time. As a quick test:
tic;for ii=1:10^10;end;toc
Time elapsed = 19 seconds. Just going through the loop counter takes a while. I would image that even the simplest of function could take a very long time.
There might be a trade-off between speed and memory consumption. That being said, you have several alternatives to make your code faster. The first step is to use the profiler to find bottlenecks in your code and optimize the functions that are taking the most time. After doing that, you have several alternatives
  • Parallelize your code (doc parfor, parallel computing toolbox and others)
  • Get a faster computer
  • Use a lower level language, such as C/C++ or Fortran that are bound to be much faster than Matlab.

Sign in to comment.

More Answers (1)

Jan
Jan on 16 May 2013
Edited: Jan on 3 Aug 2015
"Out of memory" means, that you try to create an array, which does not fit into the available free memory.
The solutions are trivial:
  1. Install more memory (twice as much is a good strategy)
  2. Because RAM is cheap even more memory is fine
  3. Switch off other applications, which occupy memory
  4. Clear unused variables in Matlab
  5. Use smaller data types if possible (a double needs 8 bytes, a uint16 only 2)
  6. Use a 64-bit system and Matlab, such that you can use the bunch of gigabytes you have installed according to the points 1. and 2.
You find exactly the same answers, when you search in the forum for "Out of memory".
[EDITED]
Another problem might be, that you try to create a 10 dimensional ndgrid. I'm not sure why you use matrices as input to ndgrid, because I'd expect vectors -- but this might be meaningful for reasons I do no know. But 10 dimensional regular grids tends to be extremely huge. If each dimension has 10 elements, Matlab needs to store 10^10 elements. So I guess, that there is a design problem and you want to do something else.

Categories

Find more on Startup and Shutdown in Help Center and File Exchange

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!