Friday, 13th December - 11am Willem Zuidema: Seminar Title: Under the hood: what LLMs and NSMs learn about our language, and what they teach us about usAbstract: Large Language Models (LLMs) and Neural Speech Models (NSMs) have made big advances in the last few years in their abilities to mimic and process human language and speech. Their internal representations, however, are notoriously difficult to interpret, limiting their usefulness for cognitive and neuroscience. However, a new generation of posthoc interpretability techniques, based on causal interventions, provide an increasingly detailed look under the hood. These techniques allow us, in some cases, to reveal the nature of the learned representations, assess how general the learned rules are and formulate new hypotheses on how humans might process aspects of language and speech. I will discuss examples from syntax and phonology, and speculate on the future impact of AI-models on the cognitive science of language.Bio:Willem Zuidema (a.k.a. Jelle) is associate professor of NLP and Explainable AI at the Institute for Logic, Language & Computation (ILLC). He obtained his PhD from the University of Edinburgh (2005), with a dissertation on computational models of language evolution. At ILLC, he has done pioneering work in deep learning for NLP (since 2012) and interpretability methods for LSTMs and Transformers (since 2016). He leads the InDeep consortium, focused on interpretability for text and sound, involving 7 PhD students and 5 universities in the Netherlands. He teaches in the Master programs Brain & Cognitive Sciences and Artificial Intelligence. Dec 13 2024 11.00 - 12.00 Friday, 13th December - 11am Willem Zuidema: Seminar This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk. IF G.03 and on Teams Contact
Friday, 13th December - 11am Willem Zuidema: Seminar Title: Under the hood: what LLMs and NSMs learn about our language, and what they teach us about usAbstract: Large Language Models (LLMs) and Neural Speech Models (NSMs) have made big advances in the last few years in their abilities to mimic and process human language and speech. Their internal representations, however, are notoriously difficult to interpret, limiting their usefulness for cognitive and neuroscience. However, a new generation of posthoc interpretability techniques, based on causal interventions, provide an increasingly detailed look under the hood. These techniques allow us, in some cases, to reveal the nature of the learned representations, assess how general the learned rules are and formulate new hypotheses on how humans might process aspects of language and speech. I will discuss examples from syntax and phonology, and speculate on the future impact of AI-models on the cognitive science of language.Bio:Willem Zuidema (a.k.a. Jelle) is associate professor of NLP and Explainable AI at the Institute for Logic, Language & Computation (ILLC). He obtained his PhD from the University of Edinburgh (2005), with a dissertation on computational models of language evolution. At ILLC, he has done pioneering work in deep learning for NLP (since 2012) and interpretability methods for LSTMs and Transformers (since 2016). He leads the InDeep consortium, focused on interpretability for text and sound, involving 7 PhD students and 5 universities in the Netherlands. He teaches in the Master programs Brain & Cognitive Sciences and Artificial Intelligence. Dec 13 2024 11.00 - 12.00 Friday, 13th December - 11am Willem Zuidema: Seminar This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk. IF G.03 and on Teams Contact
Dec 13 2024 11.00 - 12.00 Friday, 13th December - 11am Willem Zuidema: Seminar This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk.
