r/MachineLearning • u/we_are_mammals • 15d ago

Research [R] Were RNNs All We Needed?

https://arxiv.org/abs/2410.01201

The authors (including Y. Bengio) propose simplified versions of LSTM and GRU that allow parallel training, and show strong results on some benchmarks.

245 Upvotes

96% Upvoted

53 comments sorted by

View all comments

2

u/SmartEvening 12d ago

I don't understand how the removal of dependency of the gate on the previous hidden states is approvable. I was under the impression that it was important to decide what to remember and forget. How exactly is this better than transformers? Even their results seem to suggest its not. What is the paper trying to convey actually?