Wednesday, 4th December - 11am Sohee Yang: Seminar Title: Latent Multi-Hop Reasoning of Large Language Models Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in complex reasoning tasks, but it remains unclear whether they latently perform multi-hop reasoning when processing queries like "The mother of the singer of 'Superstition' is." Understanding whether and how well LLMs perform latent multi-hop reasoning is crucial as it can provide us with implications in the parameter efficiency, knowledge controllability, and compositional generalization ability of LLMs. In this talk, I will present three studies that systematically investigate this question: (1) evidence for the existence of latent reasoning pathways, (2) a deeper analysis of the mechanism behind these pathways, and (3) a rigorous evaluation of LLMs' latent multi-hop reasoning ability while minimizing their chance of exploiting shortcuts. The findings from these works reveal both promising capabilities and significant limitations in current LLMs' latent reasoning, offering important insights for future improvements. Bio: Sohee Yang is a second-year Ph.D. student at UCL and a part-time research scientist intern at Google DeepMind, splitting her time between the two organizations during her Ph.D. studies. She is co-advised by Prof. Pontus Stenetorp and Prof. Sebastian Riedel at UCL NLP Group, with Prof. Sebastian Riedel and Prof. Mor Geva advising her on GDM projects. Her research focuses on natural language processing and machine learning, with particular emphasis on understanding and enhancing the reasoning abilities of NLP/ML systems in a safe and controllable way. She completed her Master's in Artificial Intelligence at Kim Jaechul Graduate School of AI, KAIST, advised by Prof. Minjoon Seo at Language & Knowledge Lab. Prior to her graduate studies, she was a research engineer at Naver Clova for 2.5 years. Dec 04 2024 11.00 - 12.00 Wednesday, 4th December - 11am Sohee Yang: Seminar This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk. Seminar room 5, Crystal Macmillan Building Contact
Wednesday, 4th December - 11am Sohee Yang: Seminar Title: Latent Multi-Hop Reasoning of Large Language Models Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in complex reasoning tasks, but it remains unclear whether they latently perform multi-hop reasoning when processing queries like "The mother of the singer of 'Superstition' is." Understanding whether and how well LLMs perform latent multi-hop reasoning is crucial as it can provide us with implications in the parameter efficiency, knowledge controllability, and compositional generalization ability of LLMs. In this talk, I will present three studies that systematically investigate this question: (1) evidence for the existence of latent reasoning pathways, (2) a deeper analysis of the mechanism behind these pathways, and (3) a rigorous evaluation of LLMs' latent multi-hop reasoning ability while minimizing their chance of exploiting shortcuts. The findings from these works reveal both promising capabilities and significant limitations in current LLMs' latent reasoning, offering important insights for future improvements. Bio: Sohee Yang is a second-year Ph.D. student at UCL and a part-time research scientist intern at Google DeepMind, splitting her time between the two organizations during her Ph.D. studies. She is co-advised by Prof. Pontus Stenetorp and Prof. Sebastian Riedel at UCL NLP Group, with Prof. Sebastian Riedel and Prof. Mor Geva advising her on GDM projects. Her research focuses on natural language processing and machine learning, with particular emphasis on understanding and enhancing the reasoning abilities of NLP/ML systems in a safe and controllable way. She completed her Master's in Artificial Intelligence at Kim Jaechul Graduate School of AI, KAIST, advised by Prof. Minjoon Seo at Language & Knowledge Lab. Prior to her graduate studies, she was a research engineer at Naver Clova for 2.5 years. Dec 04 2024 11.00 - 12.00 Wednesday, 4th December - 11am Sohee Yang: Seminar This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk. Seminar room 5, Crystal Macmillan Building Contact
Dec 04 2024 11.00 - 12.00 Wednesday, 4th December - 11am Sohee Yang: Seminar This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk.
