Info

This question is closed. Reopen it to edit or answer.

Hadoop job fails with unable to find DCS

2 views (last 30 days)
Cameron Taggart
Cameron Taggart on 1 Jun 2016
Closed: MATLAB Answer Bot on 20 Aug 2021
Here is the error:
>> cluster.HadoopProperties('ClusterMatlabRoot') = '/usr/local/MATLAB/R2016a/toolbox/distcomp';
>> meanDelay = mapreduce(ds,@meanArrivalDelayMapper,@meanArrivalDelayReducer,mr,'OutputFolder',outputFolder)
Parallel mapreduce execution on the Hadoop cluster:
********************************
* MAPREDUCE PROGRESS *
********************************
Map 0% Reduce 0%
Error using mapreduce (line 101)
The HADOOP job failed to complete.
Caused by:
The HADOOP job was not able to find a correct installation of MATLAB
Distributed Computing Server for attempt 1 of 'MAP' task 0. Ensure that the
'ClusterMatlabRoot' property on the mapreducer cluster object is set to the
MATLAB Distributed Computing Server installation on the cluster.
I've installed MDCS on the master node and both worker nodes. Am I setting the wrong property? Am I setting it to the wrong value? What do I need to do to get past this error?
  1 Comment
Jeevan Joishi
Jeevan Joishi on 7 Jun 2016
Edited: Jeevan Joishi on 7 Jun 2016
Hi Cameron,
Can you also include the following details in your question -
1. The version of MATLAB and MATLAB Distributed Computing Server(MDCS). I believe MATLAB and MDCS are tied to specific release as of now.
2. Did you validate MDCS setup? Are you able to ping the server from the client and vice-versa? Information on how to manually validate MDCS or with specific schedulers are given in Link1, Link2, Link3
Additional information can be found in this document.
If the issue still persists, I suggest you contact MathWorks Technical Support.

Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!