Random Numbers in Parallel Environment

Hey,

I have the problem, that in a parallel code, everytime I run it, it produces exactly the same results. Since it relies on random numbers, this should not be the case (if you are interested, it is a digital communication scheme, i.e., the information data and the noise is created randomly)! The problem is that I wrote script which stores the current results and loads them the next time I run this script. But right now, this doesn't make since, because it is like simply adding up the results from the first run (because - as I said - every run produces the same results).

To illustrate my problems, here's a abit of code

for i=1:4
   rand(2,2)
end

Execute this script a couple of times in a row - all results will look completely different. Perfect.

Now switch to parallel:

matlabpool local 4
parfor i=1:4
   rand(2,2)
end

Execute this script a couple of times in a row - everytime the same results. Bad.

I hope my problem is clear. I already found a question here (<http://www.mathworks.de/matlabcentral/answers/53183-random-number-generation-for-parallel-computing-toolbox)>, but the answer seems to be more focused on how to deal with the different streams on each worker. That is not my problem. My problem is, that every execution of this script produces the same results.

Thanks for your help.

Answer 1

[Edric Ellis]

Parallel Computing Toolbox workers very carefully set their random number generator state to the same state each time a pool is opened. This is exactly analogous to what happens when MATLAB starts - it produces the same random numbers each time you start it. For example, in R2013b, the first value returned by RAND is always 0.8147.

It sounds like what you'd like to do is either save and restore the random state from the workers so that they can continue where they left off, or else something like "rng('shuffle')". Unfortunately, "rng('shuffle')" is not a good choice in parallel since it's based on the current time - so there's a strong likelihood of workers getting the same random state.

Here's a demonstration of how you could save and then restore the random state on each worker which will let you continue your experiment where you left off:

    % Step 1 - observe the first two random numbers on each worker
    pool = parpool('local', 3);
    spmd, rand, end
    spmd, rand, end
    delete(pool);

    % Step 2 - open the pool again, see that we get the same numbers
    pool = parpool('local', 3);
    spmd, rand, end

    % Step 3 - grab the RNG state from each worker
    spmd, state = rng(); end
    % convert 'state' to a local cell array
    state = state(:);

    % Step 4 - close and re-open the pool, then restore the state
    delete(pool);
    pool = parpool('local', 3);
    % Each worker uses 'labindex' to pick their own state
    spmd, rng(state{labindex}); end

    % Step 5 - observe random numbers - should match second output
    % from Step 1
    spmd, rand, end

Answer 2

[Sean de Wolski]

Does every execution create the same results if you do not reinitialize the matlabpool by closing and opening it?

Unable to reproduce:

C = cell(4,1);
C2 = C;
C3 = C;
gcp
parfor ii = 1:4
    C{ii} = rand(2,2);
end

parfor ii = 1:4
    C2{ii} = rand(2,2);
end

delete(gcp);
gcp

parfor ii = 1:4
    C3{ii} = rand(2,2);
end

isequal(C,C2)
isequal(C2,C3)
isequal(C,C3)

All falses.

[Christoph]

Thanks for your input. In this case it works - but only as long as the matlabpool is open, then it starts again.

The problem is this: As I wrote above, I want it to use to save the current simulation state. Now imagine the following scenario: I start the simulation, pause it, close Matlab, open it the next day, continue the simulation. In this case, once again, I produce the exact same output.

What I did now is a

parfor i=1:4
    rng('shuffle')
    % calculations
end

This way, all workers produce the exact same numbers, which in my case is not too bad, but I don't feel like it's an elegant solution either.

[Sean de Wolski]

rng provides an output for the current state of the random number generator. Store this output and reinitialize the random number generator there on the next run.