Mamba & Jamba

Published: April 24, 2024, 3:45 p.m.

b"First there was Mamba... now there is Jamba from AI21. This is a model that combines the best non-transformer goodness of Mamba with good 'ol attention layers. This results in a highly performant and efficient model that AI21 has open sourced! We hear all about it (along with a variety of other LLM things) from AI21's co-founder Yoav."