RWKV: reinventing RNNs for the transformer era
The Fifth Elephant paper reading meet-up - April 2024
Apr 2024
1 Mon
2 Tue
3 Wed
4 Thu
5 Fri 05:30 PM – 07:00 PM IST
6 Sat
7 Sun
Pinned update
Parking and security details for the papers’ reading session This update is for participants only
In the last three years, RNNs have caught up with the unparalleled capabilities of Transformers. The promise of Receptive Weighted Key Value (RWKV) is that this novel architecture combines the desirable aspects of both RNNs and Transformers: the massively parallelizable transformer-esque training, and RNN’s consistent computational and memory complexity during inference.
RWKV (pronounced as “RwaKuv”) is an attention-free language model, theoretically capable of handling an “infinite” context length.
Yashodeep Deshmukh is Deputy Manager at Ashok Leyland.
This paper discussion will be held at Atlassian’s office in EGL, Bangalore. In-person attendance is free. The Fifth Elephant members can join remotely to watch the live stream.
The monthly discussions are organized to understand popular papers in Generative AI, DL, and ML domains. Papers are curated to benefit the community. The paper discussion is organized on the first Friday of each month, from 5:30 PM - 7:00 PM.
For inquiries, leave a comment or call The Fifth Elephant at +91-7676332020.
Hosted by
Supported by
Venue host
Hosted by
Supported by
Venue host