Friday, 24th February 2023 - 11am Patrick Lewis : Seminar

Title: “Atlas: Few-shot Learning with Retrieval Augmented Language Models."

Abstract:

Large language models have shown impressive few-shot results on a wide range of tasks. However, when knowledge is key for such results, as is the case for tasks such as question answering and fact checking, massive parameter counts to store knowledge seem to be needed. Retrieval augmented models are known to excel at knowledge intensive tasks without the need for as many parameters, but it is unclear whether they work in few-shot settings.

In this presentation, we will present Atlas, a pre-trained retrieval augmented language model able to learn knowledge intensive tasks with very few training examples. Unlike very large language models like GPT3, Gopher, and PALM, ATLAS can be easily updated with new knowledge on-the-fly at test time without needing retraining, and is much more interpretable. Finally, when using only 64 examples, ATLAS outperforms a 540B parameters model by 3% on question answering despite having 50x fewer parameters.

Bio:

Patrick Lewis is a Natural Language Processing Research Scientist at Cohere. Before this, he was a research scientist at the Fundamental AI Research Lab (FAIR) at Meta AI. He completed is PhD working with Sebastian Riedel and Pontus Stenetorp, splitting his time between FAIR and University College London.

Patrick works at the intersection of information retrieval techniques (IR) and large language models (LLMs).

Patrick is interested in how to represent, store and retrieve knowledge for use in large language models. His work focuses on building more powerful, efficient, robust and update-able models that can perform well on a wide range of NLP tasks, but also specifically excel on knowledge-intensive NLP tasks such as Question Answering and Fact Checking.