Randomness is lost in parfor loop on GPU?

3 views (last 30 days)
Daigo
Daigo on 12 Apr 2022
Edited: Daigo on 15 Apr 2022
I have a code with a following structure:
% my_code_seed#.m
myoutputs1 = my_function1(myinputs1);
seed = #;
rng(seed);
parfor iter = 1:50
myouputs2 = my_gpu_function2(myoutputs1, my_function_of_Gaussian);
result(iter).name = myoutputs2;
end
save(file_name, 'result');
What I want to do is to model some random behavior of something by using this code repeatedly. What I'm doing is to copy this code and change the value of the seed. For example, the "my_code_seed1.m" uses "seed = 1", and "my_code_seed2.m" uses "seed = 2". I run these codes in a parallel way on a GPU cluster to save the computation time. However, it seems like the results from the codes are very similar, even though I'm giving different values of seed.
Do you have an idea why rng(seed); is not working properly in this case? I appreciate any help.

Answers (1)

Steven Lord
Steven Lord on 13 Apr 2022
What I'm doing is to copy this code and change the value of the seed. For example, the "my_code_seed1.m" uses "seed = 1", and "my_code_seed2.m" uses "seed = 2".
So if later on you find a bug in my_code_seed42.m you're going to go back and modify the previous 41 files? That's inefficient. Instead write the code once as a function that accepts a seed value and call that function with the various seeds as input.
As for the randomness in a parfor, see this documentation page on repeating random numbers in a parfor loop and the page on controlling randomness to which its first paragraph links.
  4 Comments
Daigo
Daigo on 13 Apr 2022
I appreciate a lot, Edric! I didn't fully understand what the client and workers are, but now it's all clear. I will take a look at the documentations.
Daigo
Daigo on 13 Apr 2022
Edited: Daigo on 15 Apr 2022
@Edric Ellis I ended up using parallel.gpu.RandStream.create to control the rng state for gpuArray inside of forloop. Here is a toy example:
function [r, time] = example_random_gpu(val_seed)
tic
% Example data to be used in parfor loop
n = 100;
data = reshape(1:n^2, [n, n]);
% Create random streams for gpuArray
streams = parallel.gpu.RandStream.create('Threefry',...
'NumStreams', n,...
'Seed', val_seed,...
'CellOutput', true);
r = zeros(3,n,'gpuArray');
% Loop
parfor ii = 1:n
a = sum(data(ii,:))/n;
r(ii,:) = func_random(a, n, streams{ii});
end
% Get the elapsed time
time = toc;
end
function c = func_random(a, n, stream)
c = randn(stream, 1, n) + 1i*a*randn(stream)*gpuArray(linspace(-2,2,n));
end
The function example_random_gpu performs as exactly desired - the same r for the same val_seed, but a different r for a different val_seed.
However, I have another question regarding the data access. In this toy example, I have multiple parfor-loops accessing the same constant: data. Is there anyway to allow each worker to access the value of data more efficiently? I tried doing the following but the computation became much slower:
function [r, time] = example_random_gpu2(val_seed)
tic
% Example data to be used in parfor loop
n = 100;
data = reshape(1:n^2, [n, n]);
C = parallel.pool.Constant(data); % <----- ADDED
% Create random streams for gpuArray
streams = parallel.gpu.RandStream.create('Threefry',...
'NumStreams', n,...
'Seed', val_seed,...
'CellOutput', true);
r = zeros(3,n,'gpuArray');
% Loop
parfor ii = 1:n
a = sum(C.Value(ii,:))/n; % <---- Modified
r(ii,:) = func_random(a, n, streams{ii});
end
% Get the elapsed time
time = toc; % ---> Computation became slower ...
end

Sign in to comment.

Categories

Find more on Parallel for-Loops (parfor) in Help Center and File Exchange

Tags

Products


Release

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!