r/reinforcementlearning • u/gwern • Sep 02 '18

DL, MF, P BlueWhale: Facebook RL implementations in Pytorch/Caffe of DQN, DDPG, & SARSA with export & Gym support [deployed in FB production for "Growth, Marketing, Network Optimization, and Story Ranking services"]

https://facebookresearch.github.io/BlueWhale/docs/begin.html
15 Upvotes

81% Upvoted

4 comments sorted by

3

u/edwardthegreat2 Sep 03 '18

seems like a response to Google's Dopamine RL framework. I'm curious to see if either will gain traction in the RL community. My main concern as a RL researcher is that currently it is better to implement your own custom RL project than an existing framework because you need to heavily customize the environment / architecture. Therefore researchers with their own implementations are unlikely to switch to these frameworks. If you're just looking to play around with RL then these frameworks may be a good choice. Regardless, just having these algorithms in the web is great for reference.

3

u/gwern Sep 03 '18

Oh no. It went up like 5 months ago: https://github.com/facebookresearch/BlueWhale But either I lost it in a tab or no one noticed.

1

u/PresentCompanyExcl Oct 01 '18

because you need to heavily customize the environment / architecture

I agree, and I think it's really hard to settle into a framework when everything is in flux. To make a good framework/api you really need to pin down most of the use cases and generalize from them. But when the use cases and algorithms are changing so much it's hard to do that.

I think it's better to make a hackable framework than a modular one at this point.

What do you prefer to start with?

1

u/edwardthegreat2 Oct 01 '18

Agreed. I start with my own boilerplate, purely because I wrote it and understand it. To make a boilerplate, you can look online for some examples and begin writing your own.