Connection between Matlab and Unity3D

522 views (last 30 days)
BAA on 8 Apr 2015
Answered: sangmin yang on 26 Nov 2021
I need to integrate GA optimization of matlab with the game engine tool of Unity3D. Now what I need is to generate the solutions using matlab and send it to Unity3D. Unity3D will calculate the objective function and will return the results again to matlab in order generate the next solutions and so on. Is this process possible? If so, could you please support me with any information or documents that can guide my work?
  1 Comment
Les on 4 Oct 2016
I second this, has anyone solved this? I really need a way to send data between matlab and Unity3D.

Sign in to comment.

Answers (7)

Larasmoyo Nugroho
Larasmoyo Nugroho on 9 Jun 2017
Second, unity as server and Matlab as client
Unity C# code
using UnityEngine;
using System.Collections;
using System.Net;
using System.Net.Sockets;
using System.Linq;
using System;
using System.IO;
using System.Text;
public class readSocket : MonoBehaviour {
// Use this for initialization
TcpListener listener;
String msg;
void Start () {
listener=new TcpListener (55001);
listener.Start ();
print ("is listening");
// Update is called once per frame
void Update () {
if (!listener.Pending ())
print ("socket comes");
TcpClient client = listener.AcceptTcpClient ();
NetworkStream ns = client.GetStream ();
StreamReader reader = new StreamReader (ns);
msg = reader.ReadToEnd();
print (msg);
//Matlab code
clear all
tcpipClient = tcpip('',55001,'NetworkRole','Client');
a='yah!! we could make it';
in each implementation the server should runs first before the client otherwise you will get the bellow error
Connection refused:in Matlab or remote machine actively refuse the connection in unity

Larasmoyo Nugroho
Larasmoyo Nugroho on 9 Jun 2017
Edited: Larasmoyo Nugroho on 9 Jun 2017
First*, Matlab as server and unity as client
// matlab code
clear all
tcpipServer = tcpip('',55000,'NetworkRole','Server');
data = membrane(1);
rawData = fread(tcpipServer,14,'char');
for i=1:14 rawwData(i)= char(rawData(i));
// Unity C# code
using System.Collections;
using System.Net;
using System.Net.Sockets;
using System;
using System.IO;
public class Socket : MonoBehaviour {
// Use this for initialization
internal Boolean socketReady = false;
TcpClient mySocket;
NetworkStream theStream;
StreamWriter theWriter;
StreamReader theReader;
String Host = "localhost";
Int32 Port = 55000;
void Start () {
setupSocket ();
Debug.Log ("socket is set up");
// Update is called once per frame
void Update () {
public void setupSocket() {
try {
mySocket = new TcpClient(Host, Port);
theStream = mySocket.GetStream();
theWriter = new StreamWriter(theStream);
socketReady = true;
writeSocket("yah!! it works");
Debug.Log ("socket is sent");
catch (Exception e) {
Debug.Log("Socket error: " + e);

Pavel Tikhonov
Pavel Tikhonov on 28 Jan 2018
Edited: Pavel Tikhonov on 28 Jan 2018
For those who have a syntax error in "writeSocket", replace this line with:
Byte[] sendBytes = Encoding.UTF8.GetBytes("yah!! it works");
mySocket.GetStream().Write(sendBytes, 0, sendBytes.Length);

Michael Ranis
Michael Ranis on 22 Jul 2017
Edited: Walter Roberson on 22 Jul 2017
Building off of the Answer by Larasmoyo Nugroho on 9 Jun 2017
In the Matlab as Server and Unity as Client section
There is a syntax error in the C# script:
writeSocket("yah!! it works");
Here is a link that may be useful and the code associated with it. I have not gotten it to work myself (yet).
static void Connect(String server, String message)
// Create a TcpClient.
// Note, for this client to work you need to have a TcpServer
// connected to the same address as specified by the server, port
// combination.
Int32 port = 13000;
TcpClient client = new TcpClient(server, port);
// Translate the passed message into ASCII and store it as a Byte array.
Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);
// Get a client stream for reading and writing.
// Stream stream = client.GetStream();
NetworkStream stream = client.GetStream();
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
Console.WriteLine("Sent: {0}", message);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
Console.WriteLine("Received: {0}", responseData);
// Close everything.
catch (ArgumentNullException e)
Console.WriteLine("ArgumentNullException: {0}", e);
catch (SocketException e)
Console.WriteLine("SocketException: {0}", e);
Console.WriteLine("\n Press Enter to continue...");

ivan camponogara
ivan camponogara on 2 Jun 2019
Just an Update:
I am very new to unity, but i just discovered that, if you want to use Matlab as client and unity as server:
1) Your unity C# code works only if you name it "readSocket" (as specified at line 10 of the code before the "MonoBehaviour" command.
2) In the TcpListener command (line 20 of the code) within the parenthesis you have to specify the ip number in this way: IPAddress.Parse(""). Then you have to write the server number (55001) separated by a comma.
listener = new TcpListener(IPAddress.Parse(""), 55001);
3) The C# code in unity works only once you drag the C# file on the top the "Main Camera" file that you can find in the "Sample scene" file.
Hope this could help

Vittorio Palma
Vittorio Palma on 10 Dec 2019
Hello everyone, I hope this post will be read again.
I would love to ask you an help with my project. Actually I'm trying to share thermal images from Thermal Camera FLIR in HoloLens. Well this I think could happen via Unity3D. So my question is how could I share Thermal images from FLIR into unity3D? This is the step I miss, because then from Unity (where I think I would build a RawImage, I can deploy in HoloLens the view.
I hope is clear what I want to do... Is there anyone that could help? Maybe with MATLAB (but actually Im not very expert) we could reach this???
Waiting for an answer please.
  1 Comment
Rus Gabriela
Rus Gabriela on 12 Nov 2021
Hello! I woud like to know if you resolved this problem because I want to do something similar...

Sign in to comment.

sangmin yang
sangmin yang on 26 Nov 2021
Hello, when it comes to the case of Unity as a server, Matlab as a client... it works well,,,
However, When the the case of Unity as a client, Matlab as a server, I can not read the comment the msg in matlab
How can I check it?

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!