Would I able to load a Gazebo world as a training model into rlSimulinkEnv function ?

Hi, I'm trying to use Simulink to apply turtlebot3 to do deep reinforcement learning simulation, the tutorial I'm following is from Robotics Arena DRL for Walking Robots.

I face a problem is that I want to load the Gazebo model such like turtlebot3_world.world instead of the walking robot model, but I didn't find any tutorial about it.

Here is the code at the file createWalkingAgent2D.m

% Environment
mdl = 'walkingRobotRL2D';
load_system(mdl);
blk = [mdl,'/RL Agent'];
env = rlSimulinkEnv(mdl,blk,observationInfo,actionInfo);

If there is anyone knows it, please give me some advicess. Thank you!!

[Vishal Jadhav]

Hello Jen,

I am trying to use Matlab script to apply TB3 to deep reinforcemnet learning simulation. Can you share how you defined custom gazebo world as env for this project? I have my custom world and python script for resetting world in ROS-Gazebo but I need to do that with MATLAB.

Answer 1

[Cam Salzberger]

Hello Jen-Yu,

Loading a Simulink model that simulates a robot is very different than loading a Gazebo world. There is currently no capability for loading a Gazebo model inside of Simulink and accessing data from it directly.

You can, however, run Gazebo externally and load the model into that. You can then communicate with Gazebo using Gazebo Co-Simulation, ROS, or ROS 2, and feed the data received from that into your reinforcement learning algorithm. Gazebo Co-Simulation will provide a higher level of control over the Gazebo simulation, and may have more consistent performance. It may be easier to get started with ROS or ROS 2, though, since TurtleBot3 is set up to use it already.

-Cam

[Jen-Yu Lee]

Hi Mr. Salzberger,

I appreciate that you answered my question! This positive answer makes me regain my confidence when I am feeling dispired from my research.

In fact, I've succeeded in runing Gazebo and Matlab together by using ROS, I can do the topic subscribe, publish or something.

And, I have tried to use Gazebo Co-Simulation and ROS on Simulink platform, but I am confused in if I run the DDPG, it still need to finish the setup of DDPG in the .m file.

The definition of rlSimulinkEnv function is

env = rlSimulinkEnv(mdl,agentBlocks,obsInfo,actInfo)

If I have no any model, what should I fill inside?

Could you guide me some direction about it? Or if you know some tutorials about "run Gazebo externally and load the model into that" in detail? Please let me know, thank you!!!

[Emmanouil Tzorakoleftherakis]

If you don't use Simulink, you don't need to use rlSimulinkEnv. But you would need to create a custom environment in MATLAB following directions here.

[Jen-Yu Lee]

Hi Emmanouil Tzorakoleftherakis, thank you for your answer, I'll try it.