check parameter used by function with large memory

6 views (last 30 days)
Hi:
I ran my code and found that after a certain time the memory is occupied by nearly 50G. I want to find the parameters which used so much memory but there are lots of sub-functions.
is there anyway to find the corresponding parameters, sub-functions, and memory used? so that I can optimize my code?
Thanks! Yu
  3 Comments
Jan
Jan on 30 Jul 2018
How do you determine the amount of "occupied" memory? Maybe all you see is the reserved memory, which does not necessarily mean, that it is in use.
Beside the parsed source code, functions do not "use" any memory at all, except if you store persistent memory. Only data occupy memory.
Without seeing the code, it is impossible to suggest an improvement.
Yu Li
Yu Li on 30 Jul 2018
Hi: The code is a large ‘for’ loop. At the very beginning the memory is small but after some time I see 50G memory used by Marlab in ‘task manager’. So I think some parameters in the loop are accumulate during the calculation.
Thanks! Yu

Sign in to comment.

Accepted Answer

OCDER
OCDER on 30 Jul 2018
Edited: OCDER on 1 Aug 2018
REAL NEW ANSWER
You are generating a ton of invisible figure handles, and clearing the variable name without closing the figures. This will accumulate a ton of figure handles that are not referred to by any variable name (hence whos fail to detect these. Use close and not clear.
for j = 1:10
f = figure('visible', 'off');
clear f %Clears the variable, but not the handle! Use "close(f)"!
end
findobj(0, '-class', 'matlab.ui.Figure') %You have tons of figures, hidden.
close all %closes all your figures
findobj(0, '-class', 'matlab.ui.Figure') %All the figures are deleted now.
NEW ANSWER
Use the undocumented memory profiler as shown here: http://undocumentedmatlab.com/blog/undocumented-profiler-options
To summarize, do this in you matlab command window:
profile -memory on
setpref('profiler', 'showJitLines', 1);
profview
%In profview, on the top next to "Run this code:", type in the command to run your code.
OLD ANSWER
Use whos to determine the variable names and size in the for loop. Then print out the largest var and its size. Here's an example.
for j = 1:100 %Your large for loop
... blah blah blah
if mod(j, 10) == 0 %just so you don't print too many times
W = whos;
[MaxByte, MaxIdx] = max([W.bytes]);
fprintf('Largest Var = %s [%0.3f GB]\n', W(MaxIdx(1)).name, MaxByte/1e9);
end
end
  3 Comments
OCDER
OCDER on 31 Jul 2018
Interesting... just found a undocumented memory profiler. See edited New Answer above.
Yu Li
Yu Li on 1 Aug 2018
Hi:
Thanks for your reply. The problem was solved.
meanwhile, as f is closed at each loop, the performance of the code looks improved.
Thanks!
Yu

Sign in to comment.

More Answers (2)

Jan
Jan on 31 Jul 2018
Remember, that the RAM shown in the TaskManager is not necessarily "occupied". If Matlab reserves 100MB frequently in a loop, the released memory is not necessarily given back to the operating system. And if it is, the OS might not find the time currently to overwrite it with zeros, such that it can be delivered to an application again. This means, that the TaskManager does not tell you, how much RAM the code needs. The code might run with 1GB RAM also fluently, although the memory managers of Matlab and the OS reserve more memory, when it is available.
  8 Comments
Yu Li
Yu Li on 1 Aug 2018
Hi:
Thanks for your reply.
1. the clear all does not work, it could not free up the memory, only restart the matlab can complete get my memory back.
2. 'the performance looks same like before', it means the even thought the code runs with 99% memory, performance looks like the same with 10% memory.
3. I've re-run the code and will let you know what shows in the profiler.
4. you are right, I used figure ('visible','off') in my loop, but I have clear the invisible figure at end of each loop, below are my source code:
for i=1:1:1000000
f = figure('visible','off');
~blabla
print(fig_name,'-djpeg','-r1200');
clear f
end
Thanks!
Yu
OCDER
OCDER on 1 Aug 2018
Updated my answer above. That's a tricky bug.

Sign in to comment.


Guillaume
Guillaume on 1 Aug 2018
Edited: Guillaume on 1 Aug 2018
I used figure ('visible','off') in my loop, but I have clear the invisible figure at end of each loop
f = figure;
clear f
No you haven't! There's a huge difference between clear and close. clear f gets rid of the variable f but the figure it points to still exist. close(f) closes the figure pointed to by f but keep f as a variable (which is now an invalid figure handle).
So it's no wonder you're low on memory, you've got all these figures open. Replace the
clear f
by
close(f);
In general, you should never use clear. clearing a variable that is created in a loop in particular serves no purpose. close on the other hand is useful.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!