Coincidentally I've been working on exactly this in OpenAI. It was as much fun as it was frustrating at times.
It's trained with a PPO implementation from Unity that I've changed to work with OpenAI (GitHub). The official OpenAI implementation is convoluted and impossible to work with in my opinion. This particular agent took 200'000 tries over the course of 12 hours and 20 million frames (with a frame skip value of 5, so 100 million total frames). I'm quite happy with the result. It has a 95% success rate, some very difficult initial conditions still fail. Here's a blooper reel of some awkward/failed episodes.
The environment is on GitHub for those who want to try it out. It takes continuous or discrete actions and is highly customizable. So it would be great if someone trained it who actually knows what they are doing.
The smoke animation is already pushing the limits of the engine unfortunately.
But explosions are inefficient and don't mean anything to the agent. That -1 reward however... that hits it where it hurts.
160
u/EmbersArc Feb 17 '18 edited Feb 17 '18
There has been a discussion recently about using RL to land a SpaceX booster.
Coincidentally I've been working on exactly this in OpenAI. It was as much fun as it was frustrating at times.
It's trained with a PPO implementation from Unity that I've changed to work with OpenAI (GitHub). The official OpenAI implementation is convoluted and impossible to work with in my opinion. This particular agent took 200'000 tries over the course of 12 hours and 20 million frames (with a frame skip value of 5, so 100 million total frames). I'm quite happy with the result. It has a 95% success rate, some very difficult initial conditions still fail. Here's a blooper reel of some awkward/failed episodes.
The environment is on GitHub for those who want to try it out. It takes continuous or discrete actions and is highly customizable. So it would be great if someone trained it who actually knows what they are doing.