Connection between Matlab and Unity3D

I need to integrate GA optimization of matlab with the game engine tool of Unity3D. Now what I need is to generate the solutions using matlab and send it to Unity3D. Unity3D will calculate the objective function and will return the results again to matlab in order generate the next solutions and so on. Is this process possible? If so, could you please support me with any information or documents that can guide my work?

[Les]

I second this, has anyone solved this? I really need a way to send data between matlab and Unity3D.

Answer 1

[Larasmoyo Nugroho]

Second, unity as server and Matlab as client

Unity C# code

using UnityEngine;
using System.Collections;
using System.Net;
using System.Net.Sockets;
using System.Linq;
using System;
using System.IO;
using System.Text;

public class readSocket : MonoBehaviour {

      // Use this for initialization
      TcpListener listener;
      String msg;

      void Start () {
          listener=new TcpListener (55001);
          listener.Start ();
          print ("is listening");
      }
      // Update is called once per frame
      void Update () {
          if (!listener.Pending ())
          {
          } 
          else 
          {
              print ("socket comes");
              TcpClient client = listener.AcceptTcpClient ();
              NetworkStream ns = client.GetStream ();
              StreamReader reader = new StreamReader (ns);
              msg = reader.ReadToEnd();
              print (msg);

          }
      }
  }

//Matlab code

clc
clear all
tcpipClient = tcpip('127.0.0.1',55001,'NetworkRole','Client');
set(tcpipClient,'Timeout',30);
fopen(tcpipClient);
a='yah!! we could make it';
fwrite(tcpipClient,a);
fclose(tcpipClient);

in each implementation the server should runs first before the client otherwise you will get the bellow error

Connection refused:in Matlab or remote machine actively refuse the connection in unity

Answer 2

[Larasmoyo Nugroho]

First*, Matlab as server and unity as client

// matlab code

clc
clear all
tcpipServer = tcpip('0.0.0.0',55000,'NetworkRole','Server');
while(1)
data = membrane(1);
fopen(tcpipServer);
rawData = fread(tcpipServer,14,'char');
for i=1:14 rawwData(i)= char(rawData(i));
end
fclose(tcpipServer);
end

// Unity C# code

 using System.Collections;
 using System.Net;
 using System.Net.Sockets;
 using System;
 using System.IO;
 public class Socket : MonoBehaviour {

    // Use this for initialization
    internal Boolean socketReady = false;
    TcpClient mySocket;
    NetworkStream theStream;
    StreamWriter theWriter;
    StreamReader theReader;
    String Host = "localhost";
    Int32 Port = 55000;
    void Start () {
        setupSocket ();
        Debug.Log ("socket is set up");
    }
    // Update is called once per frame
    void Update () {
    }
    public void setupSocket() {
        try {
            mySocket = new TcpClient(Host, Port);
            theStream = mySocket.GetStream();
            theWriter = new StreamWriter(theStream);
            socketReady = true;
            writeSocket("yah!! it works");
            Debug.Log ("socket is sent");
        }
        catch (Exception e) {
            Debug.Log("Socket error: " + e);
        }
    }
}

Answer 3

[Pavel Tikhonov]

For those who have a syntax error in "writeSocket", replace this line with:

Byte[] sendBytes = Encoding.UTF8.GetBytes("yah!! it works");
mySocket.GetStream().Write(sendBytes, 0, sendBytes.Length);

[arun sebastian]

When I relaced the code I got the error as" Assets\uniCLI.cs(37,32): error CS0103: The name 'Encoding' does not exist in the current context". Could anyone have solution for this?

[Jorge Juez]

Hi, you need to add two more libraries to those mentioned ahead in Unity. These are:

using UnityEngine;

using System.Text;

Answer 4

[Michael Ranis]

Building off of the Answer by Larasmoyo Nugroho on 9 Jun 2017

In the Matlab as Server and Unity as Client section

There is a syntax error in the C# script:

writeSocket("yah!! it works");

Here is a link that may be useful and the code associated with it. I have not gotten it to work myself (yet).

https://msdn.microsoft.com/en-us/library/system.net.sockets.tcpclient

//
static void Connect(String server, String message) 
{
  try 
  {
    // Create a TcpClient.
    // Note, for this client to work you need to have a TcpServer 
    // connected to the same address as specified by the server, port
    // combination.
    Int32 port = 13000;
    TcpClient client = new TcpClient(server, port);

      // Translate the passed message into ASCII and store it as a Byte array.
      Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);         

      // Get a client stream for reading and writing.
     //  Stream stream = client.GetStream();

      NetworkStream stream = client.GetStream();

      // Send the message to the connected TcpServer. 
      stream.Write(data, 0, data.Length);

      Console.WriteLine("Sent: {0}", message);         

      // Receive the TcpServer.response.

      // Buffer to store the response bytes.
      data = new Byte[256];

      // String to store the response ASCII representation.
      String responseData = String.Empty;

      // Read the first batch of the TcpServer response bytes.
      Int32 bytes = stream.Read(data, 0, data.Length);
      responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
      Console.WriteLine("Received: {0}", responseData);         

      // Close everything.
      stream.Close();         
      client.Close();         
    } 
    catch (ArgumentNullException e) 
    {
      Console.WriteLine("ArgumentNullException: {0}", e);
    } 
    catch (SocketException e) 
    {
      Console.WriteLine("SocketException: {0}", e);
    }

    Console.WriteLine("\n Press Enter to continue...");
    Console.Read();
  }

Answer 5

[ivan camponogara]

Just an Update:

I am very new to unity, but i just discovered that, if you want to use Matlab as client and unity as server:

1) Your unity C# code works only if you name it "readSocket" (as specified at line 10 of the code before the "MonoBehaviour" command.

2) In the TcpListener command (line 20 of the code) within the parenthesis you have to specify the ip number in this way: IPAddress.Parse("127.0.0.1"). Then you have to write the server number (55001) separated by a comma.

listener = new TcpListener(IPAddress.Parse("127.0.0.1"), 55001);

3) The C# code in unity works only once you drag the C# file on the top the "Main Camera" file that you can find in the "Sample scene" file.

Hope this could help

Answer 6

[sangmin yang]

Hello, when it comes to the case of Unity as a server, Matlab as a client... it works well,,,

However, When the the case of Unity as a client, Matlab as a server, I can not read the comment the msg in matlab

How can I check it?

[jh]

Update your matlab function like “tcpserver”

Answer 7

[Vittorio Palma]

Hello everyone, I hope this post will be read again.

I would love to ask you an help with my project. Actually I'm trying to share thermal images from Thermal Camera FLIR in HoloLens. Well this I think could happen via Unity3D. So my question is how could I share Thermal images from FLIR into unity3D? This is the step I miss, because then from Unity (where I think I would build a RawImage, I can deploy in HoloLens the view.

I hope is clear what I want to do... Is there anyone that could help? Maybe with MATLAB (but actually Im not very expert) we could reach this???

Waiting for an answer please.

Thanks.

Regards.

[Rus Gabriela]

Hello! I woud like to know if you resolved this problem because I want to do something similar...

Answer 8

[JUDITH NJOKU]

I was wondering if anyone tried this for just integer and not 3D images. What if I wanted to establish socket communication between MATLAB and Unity. I want to visualize my MATLAB output in Unity.