Friday, 28th February - 11am Fabio Zanzotto : Seminar

Title: Memorization or Generalization? Exploring Transformer-based Large Language Models and, possibly, novel approaches 

 

Abstract: Transformer-based Large Language Models demonstrate extraordinary capabilities and, thus, change the approach of the ML/NLP/NN communities when conducting research. Large chunks of research topics are neglected as Transformer-based LLMs seem to be the ultimate solution. However, it is already emerging that a large part of the capabilities of LLMs depends on their ability to memorize. Moreover, the necessity for deep neural networks to memorize long-tailed data to obtain close to optimal generalization error has attracted a lot of discussion. In this talk, we aim to report our experience in the florid research area on LLMs, exploring how these models memorize and how they generalize from training data. 

 

Bio: Prof. Fabio Massimo Zanzotto is an associate professor at the University of Rome Tor Vergata coordinating the group of Human-centric ART. He specialized in artificial intelligence (AI) and natural language processing (NLP). Since 1998, he has been actively engaged in AI research, focusing on ethical considerations, AI applications in tourism and healthcare, and fundamental AI concepts. He coordinates and coordinated many research projects, including the European H2020 KATY project and the national Social Tourism e-Platform (STEP), Class-tAIs, and SfidaNow. Zanzotto also oversees collaborations with companies in the natural language processing domain, showcasing his significant impact on both academia and industry. Zanzotto's contributions include work on automatic textual entailment recognition, syntactic parsing, and the application of distributed and distributional models to language syntax and semantics. With over 150 publications in international and national forums, Zanzotto is a prominent figure in the field. He plays an active role in major conference committees (ACL, NAACL, EACL, EmNLP, CoLing, LREC, IJCAI, ECAI, CLEF) and serves as a reviewer for esteemed international journals. He is a member of the Association for Computational Linguistics (ACL) and the Italian Association for Artificial Intelligence and is a founding member of the Italian Association for Computational Linguistics. Moreover, he contributed to the creation of two spin-offs Reveal SRL and DevIt SRL.