Connect CarMaker and ROS for Reinforcement Learning

Coding & Integration ROS
1

Hello My friends.

I need your help to proceed my research with CM.
What I want to do is to use neural net(NN) in python as control input(steer or trq) maker.
To do so, I need to train the NN in the python with Reinforcement learning(RL).

The necessary properties to train with RL, I think to extract the state of the car from CM and
push the control input from python each time step (or user defined time step).

And as I know, to exchange data at each time step, the c code based cycle control, CM4SLX (also controlled by python as well) and the ROS as the middleware are the supported means.

My choice is to use the ROS. But I want to know the possibility to execute RL.

  1. Is it possible to control the CM by external node?
    The node that contains NN and RL algorithm need the fully control ability of the simulation like start, restart or step.
    For example, if external node give “reset” topic to CM node, then, CM node stop the simulation and restart with initial values.

  2. Is it possible to pause the CM while train function and NN fead forward function is running in external node?
    It is extended question of above one. In the documentation of the CMRosIF, I saw the topic of synchronization.
    I think the data triggered synchronization is what i’m looking for.
    However, the doc says, the CM node just wait for the expected msg during received cycle.
    I want to make the simulation stop until it get msg from external node.

  3. Is it possible to use code generation and ROS in same project?
    In my research, the RL is some kind of compensator of the conventional controller designed on the Simulink.
    So the code generated plug in model need to be included in CM simulation.
    I know how to code generate, however, when I create the CMRosIF extended project, the code generation on simulink fails. (with no code generation information file error) Is it right conclude that the code generated plug in module is unable with ROS?

My lack of the knowledge of ROS and cpp makes my research very hardcore :frowning:
Any advices or opinions are very welcome to me.
Thank you for reading.

2

Hello,
Here are the answers in short.

  1. Yes, that should be possible with a bit of coding. However, even if it’s not possible, everything that you described could be dockerised. That means that even outside of CarMaker, you can bring down a container and spin a new one on demand.
  2. Yes, the small example provided by IPG in our FAQ pages pretty much does that simple task. You have the CarMaker ROS node and you have an “external” ROS node. The external node can send a sync signal that causes the CarMaker node to wait until the sync signals tells it to proceed.
  3. To my knowledge, currently the ROS integration with CarMaker works only as a direct plugin, and it only works in Ubuntu (not Windows). I’m not entirely certain that it can be made to work with code generated from Simulink plugins, but I am almost certain that the solution cannot be plugged back into Simulink.

The good news about ROS is that if you have a ROS node on the CarMaker side ready to publish and receive what you need, the external ROS node that communicates with it can be written in anything you want, Python, Matlab, etc. It’s just that the node embedded inside CarMaker is basically a compiled library and a C++ “plugin” for CarMaker, and there is no way to rewrite that purely in anything else at the moment.

A very good starting point for you would be the existing FSAI project that is given to all students in the Formula Student AI competition that IPG sponsors. It contains a lot of useful stuff that you can start with. It also contains documentation that is a lot more detailed. I believe that IPG is free to share this with customers on-demand, but we cannot support any changes to the project.

I strongly recommend that if your university has a Formula Student team, you get in touch with them. Apart from very quickly gaining access to the FSAI project, you can also gain a lot from their experience if they have used the FSAI CarMaker ROS node. It is possible that your research could also be useful for them to get ahead in the competition.

Regards,
Bogomil