fminunc : A VERY STRANGE PROBLEM!
16 views (last 30 days)
Show older comments
Hi
I use fminunc to solve a minimization problem. Fminunc make hundeads of calls of a simple function which I optimized for the gpu to improve the performance.
This is what happens when I make a comparison between cpu and gpu for a single call (test is external to fminuc):
TVD=tvd_sim2_mex(x,y, lam, Nit,t); % 0.018 s
TVD=tvd_sim2(x,y, lam, Nit,t); % 0.003 s 6x
As you can see the performance is 6X faster for the cpu.
The gpu profiling tells me the problem is about gpu malloc.
And this is what happen when I call the function 1000 times:
for i=1:1000
tic
TVD=tvd_sim2_mex(x,y, lam, Nit,t);
mytime(i)=toc;
end % 0.0005 s 6x
TVD=tvd_sim2(x,y, lam, Nit,t); % 0.003 s
As you can see the performance is 6X faster for the gpu.
Now....I can't know what exaclty happens inside fminunc function but surly I can say without doubt that
the only difference between the two situation is the function tvd_sim2. No modification to fminunc has been made.
The gpu always delete the memory at the end of every single call.
The function tvd_sim2 is compiled only once before fminunc.
This is what happens when I make a comparison between fminunc using tvd_sim2 and tvd_sim2_mex
(the function tvd_sim launches fminunc):
tic
[y, cost] = tvd_sim(x, lam, Nit,t); % Run whit tvd_sim2
toc
Solver stopped prematurely.
fminunc stopped because it exceeded the iteration limit,
options.MaxIterations = 5.000000e+01.
Elapsed time is 48.020835 seconds.
and:
tic
[y, cost] = tvd_sim(x, lam, Nit,t); % Run with tvd_sim2_mex
toc
Solver stopped prematurely.
fminunc stopped because it exceeded the iteration limit,
options.MaxIterations = 5.000000e+01.
Elapsed time is 179.953791 seconds.
In few words.... why does it go slower even if it go faster?
I thought....in my "1000 times for loop" the variable y is always equal but fminunc changes it every time.
This is due to the optimization.
But this is a false problem:
for i=1:1000
y=rand(4096,1);
tic
TVD=tvd_sim2_MEX_mex(x,y, lam, Nit,t);
mytime(i)=toc;
end
disp('mean time:');
disp(mean(mytime));
mean time:
5.5624e-04
The fact is the gpu reallocate the memory every function call,there's no difference between an equal input or a different one!
I add the screenshots of the function's run with and without the gpu. As you can see all the time is in charge of this function.
Which environmental variable (in the broad sense) can so substantially modify the performance of a gpu running the same code?
Or beyond the evidence is it not the same code?
Thanks!
23 Comments
Bruno Luong
on 9 Nov 2022
Edited: Bruno Luong
on 9 Nov 2022
If you don't provide the gradient MATLAB calls 4000-8000 time the objective to compute the gradients, if you do, then MATLAB do not need to evalutae 4000 time to estimate the gradient but get it from your function. Imagine the time you could save.
Accepted Answer
Matt J
on 7 Nov 2022
Edited: Matt J
on 7 Nov 2022
If I had to guess, the GPU cannot achieve faster speeds because fminunc requires that you pull the results of GPU computation back to the CPU after every call to the objective function. This is because fminunc has to do intermediate computations of its own which must take place on the CPU. It is plausible that the overhead of the CPU-GPU transfers, given the simplicity of your objective, is dominating the computation time.
Why some of your timing experiments do not bear this out is unclear, but as Walter says, it is not clear that your timing methods are valid. tic and toc by themselves are not reliable unless you do something to synchronize the GPU with MATLAB. You should probably be using gputimeit instead, or something in your CUDA code, I guess (__syncthreads()?).
10 Comments
More Answers (2)
Ram Kokku
on 8 Nov 2022
Edited: Walter Roberson
on 8 Nov 2022
As my colleague Hariprasad mentioned, GPU Coder is a capable of
- Allocate memory once and reuse it for subsequent calls. Use cfg.GpuConfig.EnableMemoryManager = true; to enable this.
- Take MATLAB gpuArray as input. You are doing this already. But this may not always help. for example, if GPU Coder choices to keep the first use a particular input on CPU (for some reason), it would incur an additional copy.
Further,
- you may use gpucoder.profile ( https://www.mathworks.com/help/gpucoder/ref/gpucoder.profile.html ) to find the bottlenecks.
- Use of cell arrays and structures may not play will with GPU Coder with regards to copies. consider break the cell array elements to separate variables.
- Take a look at the generated code and see GPU Coder is able to parallelize the key piece of your code.
- If you are open to share your code, I can take a quick look.
5 Comments
Bruno Luong
on 8 Nov 2022
Edited: Bruno Luong
on 9 Nov 2022
sum(abs((diffyx./ycut.^2).^2))
abs has no effect.
Both explicit casting to double is not necessary but it probably does make any harm either.
To summarize I would rather code like this (Warning: not tested)
ycut=(abs(y)-(t>0)).*y./t; % edit missing parenthesis
ycut=ycut+~ycut;
diffxx=x(2:n,1)-x(1:n-1,1);
diffxx=diffxx(2:n-1,1)-diffxx(1:n-2,1);
TVD=1/2.*sum(((y-x)./ycut.^2).^2) + lam.*sum(abs(diffxx));
Bruno Luong
on 8 Nov 2022
Edited: Bruno Luong
on 8 Nov 2022
Just shooting in the dark here and wonder if you let the UseParallel option of fminunc to true or false? It could be that the gradient computation is efficient on CPU but not on GPU depending on this option.
Also your objective function is not very differentiable with all the logical and abs, it could be that fminunc have the hard time to optimize, and time is more sentitive to the numerical truncation, that is differently with gpu-mex and cpu-matlab.
BTW the objective function is simple enough to compute analytic gradient.
11 Comments
Bruno Luong
on 9 Nov 2022
"if this can help"
Certainly I'll remember doing the wait the next tic-toc woth GPU code.
See Also
Categories
Find more on Creation of Accelerated Executable in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!