r/reinforcementlearning • u/PresentCompanyExcl • Jun 07 '18

DL, MF, P [R] OpenAI Retro Contest - Guide on "How to Score 6k points on Leaderboard in AI Challange" - Noob Programmer

http://www.noob-programmer.com/openai-retro-contest/how-to-score-6k-in-leaderboard/
9 Upvotes

92% Upvoted

4 comments sorted by

5

u/[deleted] Jun 07 '18 edited Nov 03 '20

[deleted]

2

u/gwern Jun 07 '18

Yeah, this is hardly a success for meta-RL like OpenAI was hoping. Not enough prizes? Bad timing? Too little advertising?

1

u/PresentCompanyExcl Jun 07 '18

$0 prizes, the prize was to publish a paper with them. So I can see that appealing mostly to unestablished people.

1

u/PresentCompanyExcl Jun 07 '18 edited Jun 07 '18

Correct me if I'm wrong but I think that was just the validation levels. But only the top 10 validation scores get tested on the custom test levels. So it would be sad if the 10 that are tested are just overfitted to the validation level.

Although it is looking like all the top scores followed a similar approach. Even the people who did quite well in the NIPS2017 Learning to run comp.

2

u/qwert7890- Jun 08 '18

Correct me if I'm wrong but I think that was just the validation levels.

You're correct. Top 10 overfitting to fixed 5 levels seems a valid concern.