The robot trained with Proximal Policy Optimization (PPO) algorithm learnt to walk straight after about 20000 episodes.
The reinforcement learning model was implemented with Tensorflow.
The physical model was created with PyChrono the Python Extension of Project Chrono (https://projectchrono.org/).
Link to PyChrono: https://projectchrono.org/pychrono/
Source codes: http://api.projectchrono.org/development/tutorial_pychrono_demo_tensorflow.html
Chrono has several advantages over MuJoco since, for instance, it’s free, it’s open source and can handle stiff contacts and deformable bodies.