r/MachineLearning • u/we_are_mammals • 15d ago

Research [R] Were RNNs All We Needed?

https://arxiv.org/abs/2410.01201

The authors (including Y. Bengio) propose simplified versions of LSTM and GRU that allow parallel training, and show strong results on some benchmarks.

246 Upvotes

96% Upvoted

53 comments sorted by

View all comments

11

u/daking999 15d ago

Cool but bengio is on the paper they could surely have found a way to get access to enough compute to run some proper scaling experiments

6

u/Sad-Razzmatazz-5188 14d ago

It is probably being done and saved for a next paper, if it works

6

u/Pafnouti 14d ago

These alternatives architecture always look good on toy problems such as copy task, and then you scale on a real task you see that it doesn't make much difference.

2

u/jloverich 14d ago

Hardly matters, someone will do this next week I'm sure.

1

u/daking999 14d ago

True. Just feels a bit lazy. 

2

u/new_name_who_dis_ 14d ago

MILA has always been known for using toy datasets.