RecurrentGemma: Moving Past Transformers for Efficient Open Language Models [pdf]

Code here: https://github.com/google-deepmind/recurrentgemma

Checkpoints here for both base pre-trained model and an IT version for dialogue: https://www.kaggle.com/models/google/recurrentgemma