Learn more about MATLAB Distributed Computing Server

Troubleshooting

On this page…
License Errors Verifying Multicast Communications Memory Errors on UNIX Operating Systems Running Server Processes from a Windows Network Installation Required Ports Ephemeral TCP Ports with Job Manager

This section offers advice on solving problems you might encounter with MATLAB Distributed Computing Server software.

License Errors

When starting a MATLAB worker, a licensing problem might result in the message

License checkout failed. No such FEATURE exists. 
License Manager Error -5

There are many reasons why you might receive this error:

• This message usually indicates that you are trying to use a product for which you are not licensed. Look at your license.dat file located within your MATLAB installation to see if you are licensed to use this product.

• If you are licensed for this product, this error may be the result of having extra carriage returns or tabs in your license file. To avoid this, ensure that each line begins with either #, SERVER, DAEMON, or INCREMENT.

  After fixing your license.dat file, restart your license manager and MATLAB should work properly.

• This error may also be the result of an incorrect system date. If your system date is before the date that your license was made, you will get this error.

• If you receive this error when starting a worker with MATLAB Distributed Computing Server software:

  • You may be calling the startworker command from an installation that does not have access to a worker license. For example, starting a worker from a client installation of the Parallel Computing Toolbox product causes the following error:

    The mdce service on the host hostname 
    returned the following error:
    
    Problem starting the MATLAB worker.
    
    The cause of this problem is:
    
    ==============================================================
       Most likely, the MATLAB worker failed to start due to a
       licensing problem, or MATLAB crashed during startup.  Check 
       the worker log file
       /tmp/mdce_user/node_node_worker_05-11-01_16-52-03_953.log
       for more detailed information.  The mdce log file
       /tmp/mdce_user/mdce-service.log
       may also contain some additional information.
    ===============================================================
    

    In the worker log files, you see the following information:

    License checkout failed.
    License Manager Error -15
    MATLAB is unable to connect to the license server. 
    Check that the license manager has been started, and that the 
    MATLAB client machine can communicate with the license server.
    
    Troubleshoot this issue by visiting: 
    http://www.mathworks.com/support/lme/R2009a/15
    
    Diagnostic Information:
    Feature: MATLAB_Distrib_Comp_Engine 
    License path: /apps/matlab/etc/license.dat 
    FLEXnet Licensing error: -15,570. System Error: 115
    

  • If you installed only the Parallel Computing Toolbox product, and you are attempting to run a worker on the same machine, you will receive this error because the MATLAB Distributed Computing Server product is not installed, and therefore the worker cannot obtain a license.

Back to Top

Verifying Multicast Communications

Note   Although Version 4 of the parallel computing products continues to support multicast communications between its processes, multicast is not recommended and might not be supported in future releases.

Multicast, unlike TCP/IP or UDP, is a subscription-based protocol where a number of machines on a network indicate to the network their interest in particular packets originating somewhere on that network. By contrast, both UDP and TCP packets are always bound for a single machine, usually indicated by its IP address.

The main tools for investigating this type of packet are:

• tcpdump for UNIX operating systems

• winpcap and ethereal for Microsoft Windows operating systems

• A Java™ class included with Version 3 of the parallel computing products.

The Java class is called com.mathworks.toolbox.distcomp.test.MulticastTester. Both its static main method and its constructor take two input arguments: the multicast group to join and the port number to use.

This Java class has a number of simple methods to attempt to join a specified multicast group. Once the class has successfully joined the group, it has methods to send messages to the group, listen for messages from the group, and display what it receives. The class can be used both inside MATLAB and from a call to Java software.

Inside MATLAB, use the class as follows:

m = com.mathworks.toolbox.distcomp.test.MulticastTester('239.1.1.1', 9999);
m.startSendingThread;
m.startListeningThread;
 0 : host1name : 0
 1 : host2name : 0

From a shell prompt, type (assuming that java is on your path)

java -cp distcomp.jar com.mathworks.toolbox.distcomp.test.MulticastTester 
0 : host1name : 0
1 : host2name : 0

Back to Top

Memory Errors on UNIX Operating Systems

If the number of threads created by the server services on a machine running a UNIX operating system exceeds the limitation set by the maxproc value, the services fail and generate an out-of-memory error. Check your maxproc value on a UNIX operating system with the limit command. (Different versions of UNIX software might have different names for this property instead of maxproc, such as descriptors on Solaris™ operating systems.)

Back to Top

Running Server Processes from a Windows Network Installation

Many networks are configured not to allow LocalSystem to have access to UNC or mapped network shares. In this case, run the mdce process under a different user with rights to log on as a service. See Setting the User.

Back to Top

Required Ports

Using a Job Manager

BASE_PORT.   The mdce_def file specifies and describes the ports required by the job manager and all workers. See the following file in the MATLAB installation used for each cluster process:

• matlabroot/toolbox/distcomp/bin/mdce_def.sh (on UNIX operating systems)

• matlabroot\toolbox\distcomp\bin\mdce_def.bat (on Windows operating systems)

Parallel Jobs.   On worker machines running a UNIX operating system, the number of ports required by MPICH for the running of parallel jobs ranges from BASE_PORT + 1000 to BASE_PORT + 2000.

Using a Third-Party Scheduler

Before the worker processes start, you can control the range of ports used by the workers for parallel jobs by defining the environment variable MPICH_PORT_RANGE with the value minport:maxport.

Client Ports

With the pctconfig function, you specify the ports used by the client. If the default ports cannot be used, this function allows you to configure ports separately for communication with the job manager and communication with pmode or a MATLAB pool.

Back to Top

Ephemeral TCP Ports with Job Manager

If you use the job manager on a cluster of nodes running Windows operating systems, you must make sure that a large number of ephemeral TCP ports are available on the job manager machine. By default, the maximum valid ephemeral TCP port number on a Windows operating system is 5000, but transfers of large data sets might fail if this setting is not increased. In particular, if your cluster has 32 or more workers, you should increase the maximum valid ephemeral TCP port number using the following procedure:

1. Start the Registry Editor.

2. Locate the following subkey in the registry, and click Parameters:

   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

3. On the Registry Editor window, select Edit > New > DWORD Value.

4. In the list of entries on the right, change the new value name to MaxUserPort and press Enter.

5. Right-click on the MaxUserPort entry name and select Modify.

6. In the Edit DWORD Value dialog, enter 65534 in the Value data field. Select Decimal for the Base value. Click OK.

   This parameter controls the maximum port number that is used when a program requests any available user port from the system. Typically, ephemeral (short-lived) ports are allocated between the values of 1024 and 5000 inclusive. This action allows allocation for port numbers up to 65534.

7. Quit the Registry Editor.

8. Reboot your machine.

Back to Top

Accessing Service Record Files

Admin Center

© 1984-2009- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS