Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Version 2012b breaks parallel toolbox code

Subject: Version 2012b breaks parallel toolbox code

From: Jessica Mustard

Date: 6 Mar, 2013 23:02:08

Message: 1 of 15

My parallel code stopped working with the new version 2012b, specifically I have code that grabs the ID of the current work and writes files with file names including that ID. This way I can write log files without the different workers interrupting each other.

t = getCurrentTask;
id = num2str(t.ID);
fname = [fnameroot filesep 'log_' id '.txt'];

Did this happen to anybody else? Anybody know why? Is there a more recommended method for achieving this same thing?

Thanks for your thoughts!

Subject: Version 2012b breaks parallel toolbox code

From: Edric M Ellis

Date: 7 Mar, 2013 07:44:26

Message: 2 of 15

"Jessica Mustard" <jessica.mustard@tufts.edu> writes:

> My parallel code stopped working with the new version 2012b,
> specifically I have code that grabs the ID of the current work and
> writes files with file names including that ID. This way I can write
> log files without the different workers interrupting each other.
>
> t = getCurrentTask;
> id = num2str(t.ID);
> fname = [fnameroot filesep 'log_' id '.txt'];
>
> Did this happen to anybody else? Anybody know why? Is there a more
> recommended method for achieving this same thing?

This code should continue to work in R2012b. Can you give more details
about what's going wrong, and in what context? I just tried this in
R2012b, and it worked as I expected:

>> matlabpool open local 3
>> fnameroot = 'foo';
>> spmd, t = getCurrentTask; id = num2str(t.ID); fname = ['foo' filesep 'log_' id '.log'], end
Lab 1:
  fname =
  foo/log_1.log
Lab 2:
  fname =
  foo/log_2.log
Lab 3:
  fname =
  foo/log_3.log

Cheers,

Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Jessica Mustard

Date: 7 Mar, 2013 15:50:08

Message: 3 of 15

I am making the call from within the fitness function of a genetic algorithm search running on a linux cluster. When I revert to using 2012a things run just fine.

Here is the error I receive:
Attempt to reference field of non-structure array.

Error in fitness_function (line 76)
    id = num2str(t.ID);
...

Caused by:
    Failure in initial user-supplied fitness function evaluation. GA cannot continue.

The assignment from the line immediately previous, t=getCurentTask, returns:
t =

     []


Thanks for any thoughts!

Edric M Ellis <eellis@mathworks.com> wrote in message <ytwtxon7ixx.fsf@uk-eellis0l.dhcp.mathworks.com>...
> "Jessica Mustard" <jessica.mustard@tufts.edu> writes:
>
> > My parallel code stopped working with the new version 2012b,
> > specifically I have code that grabs the ID of the current work and
> > writes files with file names including that ID. This way I can write
> > log files without the different workers interrupting each other.
> >
> > t = getCurrentTask;
> > id = num2str(t.ID);
> > fname = [fnameroot filesep 'log_' id '.txt'];
> >
> > Did this happen to anybody else? Anybody know why? Is there a more
> > recommended method for achieving this same thing?
>
> This code should continue to work in R2012b. Can you give more details
> about what's going wrong, and in what context? I just tried this in
> R2012b, and it worked as I expected:
>
> >> matlabpool open local 3
> >> fnameroot = 'foo';
> >> spmd, t = getCurrentTask; id = num2str(t.ID); fname = ['foo' filesep 'log_' id '.log'], end
> Lab 1:
> fname =
> foo/log_1.log
> Lab 2:
> fname =
> foo/log_2.log
> Lab 3:
> fname =
> foo/log_3.log
>
> Cheers,
>
> Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Edric M Ellis

Date: 7 Mar, 2013 16:59:48

Message: 4 of 15

"Jessica Mustard" <jessica.mustard@tufts.edu> writes:
> Edric M Ellis <eellis@mathworks.com> wrote in message <ytwtxon7ixx.fsf@uk-eellis0l.dhcp.mathworks.com>...
>> "Jessica Mustard" <jessica.mustard@tufts.edu> writes:
>>
>> > My parallel code stopped working with the new version 2012b,
>> > specifically I have code that grabs the ID of the current work and
>> > writes files with file names including that ID. This way I can write
>> > log files without the different workers interrupting each other.
>> >
>> > t = getCurrentTask;
>> > id = num2str(t.ID);
>> > fname = [fnameroot filesep 'log_' id '.txt'];
>> >
>> > Did this happen to anybody else? Anybody know why? Is there a more
>> > recommended method for achieving this same thing?
>>
>> This code should continue to work in R2012b. Can you give more details
>> about what's going wrong, and in what context?
>> [...]
>
> I am making the call from within the fitness function of a genetic algorithm
> search running on a linux cluster. When I revert to using 2012a things run just
> fine.
>
> Here is the error I receive:
> Attempt to reference field of non-structure array.
>
> Error in fitness_function (line 76)
> id = num2str(t.ID);
> ...
>
> Caused by:
> Failure in initial user-supplied fitness function evaluation. GA cannot continue.
>
> The assignment from the line immediately previous, t=getCurentTask, returns:
> t =
>
> []

That's strange, the behaviour of 'getCurrentTask' has not changed
(deliberately) between R2012b. It has always returned [] on the MATLAB
client, and the current task on a worker. I don't know much about the GA
search code, do you happen to know if your fitness function gets called
on the client? (Presumably things fail like this if you haven't got a
matlabpool open too?)

Cheers,

Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Jessica M

Date: 8 Mar, 2013 18:51:08

Message: 5 of 15

I have confirmed that matlabpool is open, and the fitness fcn is called on the client with 2012a. The ga function (from optimization toolbox) controls this anyways with option 'UseParallel', so I wouldn't know how to change that.

I will keep running fine with 2012a, guess no one else had this issue. Maybe it is on our end.

Thanks Edric.

Edric M Ellis <eellis@mathworks.com> wrote in message <ytwppzb6t8b.fsf@uk-eellis0l.dhcp.mathworks.com>...

> That's strange, the behaviour of 'getCurrentTask' has not changed
> (deliberately) between R2012b. It has always returned [] on the MATLAB
> client, and the current task on a worker. I don't know much about the GA
> search code, do you happen to know if your fitness function gets called
> on the client? (Presumably things fail like this if you haven't got a
> matlabpool open too?)
>
> Cheers,
>
> Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Edric M Ellis

Date: 11 Mar, 2013 08:05:22

Message: 6 of 15

"Jessica M" <noone@nowhere.edu> writes:
> Edric M Ellis <eellis@mathworks.com> wrote in message <ytwppzb6t8b.fsf@uk-eellis0l.dhcp.mathworks.com>...
>
>> That's strange, the behaviour of 'getCurrentTask' has not changed
>> (deliberately) between R2012b. It has always returned [] on the MATLAB
>> client, and the current task on a worker. I don't know much about the GA
>> search code, do you happen to know if your fitness function gets called
>> on the client? (Presumably things fail like this if you haven't got a
>> matlabpool open too?)
>>
> I have confirmed that matlabpool is open, and the fitness fcn is called on the
> client with 2012a. The ga function (from optimization toolbox) controls this
> anyways with option 'UseParallel', so I wouldn't know how to change that.
>
> I will keep running fine with 2012a, guess no one else had this issue.
> Maybe it is on our end.

Is there any chance you could post or email me a reproduction? I'd still
like to understand where the problem is - because as I say, I don't
believe anything should have changed, but clearly it has.

Cheers,

Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Kartick

Date: 24 Mar, 2013 20:21:09

Message: 7 of 15

"Jessica M" wrote in message <kh8htg$th$1@newscl01ah.mathworks.com>...
> My parallel code stopped working with the new version 2012b, specifically I have code that grabs the ID of the current work and writes files with file names including that ID. This way I can write log files without the different workers interrupting each other.
>
> t = getCurrentTask;
> id = num2str(t.ID);
> fname = [fnameroot filesep 'log_' id '.txt'];
>
> Did this happen to anybody else? Anybody know why? Is there a more recommended method for achieving this same thing?
>
> Thanks for your thoughts!

Hi,

Did anyone solve this problem? I am getting the same issue.

Subject: Version 2012b breaks parallel toolbox code

From: Edric M Ellis

Date: 25 Mar, 2013 13:31:55

Message: 8 of 15

"Kartick " <karthicks123@gmail.com> writes:

> "Jessica M" wrote in message <kh8htg$th$1@newscl01ah.mathworks.com>...
>> My parallel code stopped working with the new version 2012b, specifically I have code that grabs the ID of the current work and writes files with file names including that ID. This way I can write log files without the different workers interrupting each other.
>>
>> t = getCurrentTask;
>> id = num2str(t.ID);
>> fname = [fnameroot filesep 'log_' id '.txt'];
>>
>> Did this happen to anybody else? Anybody know why? Is there a more
>> recommended method for achieving this same thing?
>>
>> Thanks for your thoughts!
>
>
> Did anyone solve this problem? I am getting the same issue.

Hi,

Do you have any code that you can post which demonstrates the problem? I
can't reproduce it here...

Cheers,

Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Jessica M

Date: 5 Sep, 2013 16:01:08

Message: 9 of 15

Here is the code:

function x = ga_test2012b
    popsize = 10; ngens = 10;
    options = gaoptimset('UseParallel','always',...
                             'Vectorized','off',... % req for parallel
                             'PopulationSize',popsize,...
                             'Display','iter',...
                             'Generations',ngens,...
                             'EliteCount',ceil(.1*popsize),...
                             'MigrationInterval',10,...
                             'CrossoverFraction',.8);

    % start parallel processing
    matlabpool open lsfconfigJM

    % build dump path
    fnameroot = [strrep(strrep(datestr(now),':',''),' ','-')];
    mkdir(fnameroot);

    % setup fitnes function
    fitnessfcn = @(x)myfitnessfcn(x,fnameroot);

    % set bounds
    LB = -10; UB = 10;
    num_vars = length(LB);

    % run GA
    [x,fval,exitflag,output,population,scores] = ga(fitnessfcn,num_vars,[],[],[],[],LB,UB,[],options);

    function score = myfitnessfcn(x,fnameroot)
        % score
        score = x.^2;
        
        % write file
        t = getCurrentTask;
        id = num2str(t.ID); % this is where it breaks
        fname = [fnameroot filesep 'mytextfile_' id '.txt'];
        fid = fopen(fname,'a');
        fprintf(fid,'%.4f,',score);
        fclose(fid);
    end
end

Edric M Ellis <eellis@mathworks.com> wrote in message <ytwip4fwqpg.fsf@uk-eellis0l.dhcp.mathworks.com>...
> "Kartick " <karthicks123@gmail.com> writes:
>
> > "Jessica M" wrote in message <kh8htg$th$1@newscl01ah.mathworks.com>...
> >> My parallel code stopped working with the new version 2012b, specifically I have code that grabs the ID of the current work and writes files with file names including that ID. This way I can write log files without the different workers interrupting each other.
> >>
> >> t = getCurrentTask;
> >> id = num2str(t.ID);
> >> fname = [fnameroot filesep 'log_' id '.txt'];
> >>
> >> Did this happen to anybody else? Anybody know why? Is there a more
> >> recommended method for achieving this same thing?
> >>
> >> Thanks for your thoughts!
> >
> >
> > Did anyone solve this problem? I am getting the same issue.
>
> Hi,
>
> Do you have any code that you can post which demonstrates the problem? I
> can't reproduce it here...
>
> Cheers,
>
> Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Edric M Ellis

Date: 6 Sep, 2013 07:31:13

Message: 10 of 15

"Jessica M" <noone@nowhere.edu> writes:

> t = getCurrentTask;
> id = num2str(t.ID); % this is where it breaks

What's the error that you get? That works OK for me in R2012b.

Cheers,

Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Jessica M

Date: 6 Sep, 2013 13:59:08

Message: 11 of 15

Thanks for looking into this. Here is the error I receive with 2012b in full:

Attempt to reference field of non-structure array.

Error in ga_test2012b/myfitnessfcn (line 35)
        id = num2str(t.ID); % this is where it breaks

Error in ga_test2012b>@(x)myfitnessfcn(x,fnameroot) (line 20)
    fitnessfcn = @(x)myfitnessfcn(x,fnameroot);

Error in createAnonymousFcn>@(x)fcn(x,FcnArgs{:}) (line 11)
fcn_handle = @(x) fcn(x,FcnArgs{:});

Error in makeState (line 48)
            firstMemberScore = FitnessFcn(state.Population(initScoreProvided+1,:));

Error in galincon (line 18)
state = makeState(GenomeLength,FitnessFcn,Iterate,output.problemtype,options);

Error in ga (line 351)
            [x,fval,exitFlag,output,population,scores] = galincon(FitnessFcn,nvars,
            ...

Error in ga_test2012b (line 27)
    [x,fval,exitflag,output,population,scores] =
    ga(fitnessfcn,num_vars,[],[],[],[],LB,UB,[],options);

Caused by:
    Failure in initial user-supplied fitness function evaluation. GA cannot
    continue.

Edric M Ellis <eellis@mathworks.com> wrote in message <ytwtxhy9zjy.fsf@uk-eellis0l.dhcp.mathworks.com>...
> "Jessica M" <noone@nowhere.edu> writes:
>
> > t = getCurrentTask;
> > id = num2str(t.ID); % this is where it breaks
>
> What's the error that you get? That works OK for me in R2012b.
>
> Cheers,
>
> Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Jessica M

Date: 6 Sep, 2013 15:24:06

Message: 12 of 15

So getCurrentTask (and getCurrentWorker, getCurrentCluster) returns empty in Matlab 2012b, when in 2012a a task object was returned.

Could the way parallel is implemented in the genetic algorithm function or global optimization toolbox have changed in 2012b?

Thanks.

"Jessica M" wrote in message <l0cn3c$1ke$1@newscl01ah.mathworks.com>...
> Thanks for looking into this. Here is the error I receive with 2012b in full:
>
> Attempt to reference field of non-structure array.
>
> Error in ga_test2012b/myfitnessfcn (line 35)
> id = num2str(t.ID); % this is where it breaks
>
> Error in ga_test2012b>@(x)myfitnessfcn(x,fnameroot) (line 20)
> fitnessfcn = @(x)myfitnessfcn(x,fnameroot);
>
> Error in createAnonymousFcn>@(x)fcn(x,FcnArgs{:}) (line 11)
> fcn_handle = @(x) fcn(x,FcnArgs{:});
>
> Error in makeState (line 48)
> firstMemberScore = FitnessFcn(state.Population(initScoreProvided+1,:));
>
> Error in galincon (line 18)
> state = makeState(GenomeLength,FitnessFcn,Iterate,output.problemtype,options);
>
> Error in ga (line 351)
> [x,fval,exitFlag,output,population,scores] = galincon(FitnessFcn,nvars,
> ...
>
> Error in ga_test2012b (line 27)
> [x,fval,exitflag,output,population,scores] =
> ga(fitnessfcn,num_vars,[],[],[],[],LB,UB,[],options);
>
> Caused by:
> Failure in initial user-supplied fitness function evaluation. GA cannot
> continue.
>
> Edric M Ellis <eellis@mathworks.com> wrote in message <ytwtxhy9zjy.fsf@uk-eellis0l.dhcp.mathworks.com>...
> > "Jessica M" <noone@nowhere.edu> writes:
> >
> > > t = getCurrentTask;
> > > id = num2str(t.ID); % this is where it breaks
> >
> > What's the error that you get? That works OK for me in R2012b.
> >
> > Cheers,
> >
> > Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Edric M Ellis

Date: 9 Sep, 2013 07:26:03

Message: 13 of 15

"Jessica M" <noone@nowhere.edu> writes:

> So getCurrentTask (and getCurrentWorker, getCurrentCluster) returns
> empty in Matlab 2012b, when in 2012a a task object was returned.

That's definitely not expected. In R2012b and later, I only see empty
return from getCurrentTask when running on the client. Are you sure that
your matlabpool is opening correctly, or that things are running on the
workers when you think they are? You should be able to check
getCurrentTask like this:

matlabpool open local 4 % or whatever
spmd, getCurrentTask, end % should display four different tasks.

Cheers,

Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Jessica M

Date: 18 Sep, 2013 19:18:07

Message: 14 of 15

Using spmd displays different tasks from a getCurrentTask call just fine.

The problem is when getCurrentTask is called inside the fitness function of a genetic algorithm (or I assume any other global optimization fitness function) - in this case getCurrentTask returns empty.

The global optimization toolbox passes out the tasks to the different workers so I cannot control if or how things are running on workers.

If what you say is true, then the global optimization toolbox is no longer running on the workers like it should be. Please advise.

Jess

Edric M Ellis <eellis@mathworks.com> wrote in message <ytwk3iq4fsk.fsf@ukdev05glnxa64.mathworks.co.uk>...
> "Jessica M" <noone@nowhere.edu> writes:
>
> > So getCurrentTask (and getCurrentWorker, getCurrentCluster) returns
> > empty in Matlab 2012b, when in 2012a a task object was returned.
>
> That's definitely not expected. In R2012b and later, I only see empty
> return from getCurrentTask when running on the client. Are you sure that
> your matlabpool is opening correctly, or that things are running on the
> workers when you think they are? You should be able to check
> getCurrentTask like this:
>
> matlabpool open local 4 % or whatever
> spmd, getCurrentTask, end % should display four different tasks.
>
> Cheers,
>
> Edric.

Subject: Version 2012b breaks parallel toolbox code

From: Paul Kerr-Delworth

Date: 19 Sep, 2013 13:43:05

Message: 15 of 15

"Jessica M" wrote in message <l1cu9f$1a4$1@newscl01ah.mathworks.com>...
> Using spmd displays different tasks from a getCurrentTask call just fine.
>
> The problem is when getCurrentTask is called inside the fitness function of a genetic algorithm (or I assume any other global optimization fitness function) - in this case getCurrentTask returns empty.
>
> The global optimization toolbox passes out the tasks to the different workers so I cannot control if or how things are running on workers.
>
> If what you say is true, then the global optimization toolbox is no longer running on the workers like it should be. Please advise.
>
> Jess
>

Hi Jess,

From R2012b onwards, the fitness and constraint functions that are passed to GA are evaluated at a single point *on the client* before the main GA algorithm commences. This allows GA to check whether the fitness and constraint functions can be evaluated. Furthermore, this change was made to be consistent with other functions in the Optimization and Global Optimization Toolboxes.

As this evaluation is performed on the client, getCurrentTask returns empty, as you have seen. In R2012a, this check was not performed, so all function evaluations were performed on the workers and your code will run fine.

You can work around this by adding an isempty check in the fitness function, for example

% write file
t = getCurrentTask;
if isempty(t)
        id = 'client'; % or any id you like which doesn't conflict with worker ids
else
        id = num2str(t.ID); % this is where it breaks
end
[rest of your fitness function]

(I tried this with 4 workers and I get files named mytextfile_1.txt through mytextfile_4.txt and mytextfile_client.txt)

Just to note that the Optimization toolboxes require that fitness, objective and constraint functions evaluate both on the client and on the workers when UseParallel is set to 'always'. For more information, see the following link in the documentation

http://www.mathworks.com/help/optim/ug/using-parallel-computing-with-fmincon-fgoalattain-and-fminimax.html#brjhzsh

Hope this helps!

Best regards,

Paul

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us