Podcasts Technology Super Data Science: ML & AI Podcast with Jon Krohn 674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation)

674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation)

Published: April 28, 2023, 11 a.m.

b"Models like Alpaca, Vicu\\xf1a, GPT4All-J and Dolly 2.0 have relatively small model architectures, but they're prohibitively expensive to train even on a small amount of your own data. The standard model-training protocol can also lead to catastrophic forgetting. In this week's episode, Jon explores a solution to these problems, introducing listeners to Parameter-Efficient Fine-Tuning (PEFT) and the leading approach: Low-Rank Adaptation (LoRA).\\n\\nAdditional materials: www.superdatascience.com/674\\n\\nInterested in sponsoring a SuperDataScience Podcast episode? Visit JonKrohn.com/podcast for sponsorship information."