Friday, 10th February 2023 - 11am Yuxiang Wu : Seminar Title: Efficient Knowledge-intensive NLP Systems Abstract: In many important knowledge-intensive NLP tasks (e.g., question answering and dialogue), NLP systems need to exploit large-scale external knowledge in order to complete the tasks. Such systems usually require retrieving relevant knowledge first (for example, by searching Wikipedia to obtain relevant documents), and then incorporate the acquired knowledge into the model. This retrieve-then-read approach usually comes with significant computation cost, and it remains a challenging problem to efficiently represent, store, access and integrate external knowledge into models. In this talk, I will focus on three questions and some works along these directions: (1) How to reduce the computational overhead of the model after retrieving the relevant documents? We propose Adaptive Computation to reduce the computation by 4.3x while maintaining similar accuracy. (2) How to efficiently represent and store commonly-used knowledge in an external corpora? We introduce a QA generation pipeline and generated 65 million probably-asked questions, and build an efficient QA system with it. (3) How to efficiently integrate external knowledge and information into the model? We propose Efficient Memory-Augmented Transformers, which integrates external knowledge as Key-Value Memory into the models to greatly improve the accuracy in knowledge-intensive tasks. Finally, I will introduce the challenges faced by the current efficient knowledge-intensive NLP system and some future research directions. Bio: Yuxiang Wu is a final-year PhD student in the Department of Computer Science at University College London (UCL), advised by Prof. Sebastian Riedel and Prof. Pontus Stenetorp. He is going to join City University of Hong Kong as an Assistant Professor in Fall 2023. His research mainly focuses on Question Answering, Data Generation, and Knowledge-augmented Pre-trained Language Models. He has published 12 papers in top AI/NLP conferences and journals (ACL, EMNLP, AAAI, IJCAI, KDD, TACL). His works are cited by more than 1600 times, and won Best Paper Award AKBC2020, champion of Neurips 2020 EfficientQA Competition, and Best Poster Award ENLSP Workshop in NeurIPS 2022. Personal home page: https://jimmycode.github.io/ Add to your calendar vCal iCal Feb 07 2023 11.00 - 12.00 Friday, 10th February 2023 - 11am Yuxiang Wu : Seminar This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk. Informatics Forum, G.03 and online by invitation Contact
Friday, 10th February 2023 - 11am Yuxiang Wu : Seminar Title: Efficient Knowledge-intensive NLP Systems Abstract: In many important knowledge-intensive NLP tasks (e.g., question answering and dialogue), NLP systems need to exploit large-scale external knowledge in order to complete the tasks. Such systems usually require retrieving relevant knowledge first (for example, by searching Wikipedia to obtain relevant documents), and then incorporate the acquired knowledge into the model. This retrieve-then-read approach usually comes with significant computation cost, and it remains a challenging problem to efficiently represent, store, access and integrate external knowledge into models. In this talk, I will focus on three questions and some works along these directions: (1) How to reduce the computational overhead of the model after retrieving the relevant documents? We propose Adaptive Computation to reduce the computation by 4.3x while maintaining similar accuracy. (2) How to efficiently represent and store commonly-used knowledge in an external corpora? We introduce a QA generation pipeline and generated 65 million probably-asked questions, and build an efficient QA system with it. (3) How to efficiently integrate external knowledge and information into the model? We propose Efficient Memory-Augmented Transformers, which integrates external knowledge as Key-Value Memory into the models to greatly improve the accuracy in knowledge-intensive tasks. Finally, I will introduce the challenges faced by the current efficient knowledge-intensive NLP system and some future research directions. Bio: Yuxiang Wu is a final-year PhD student in the Department of Computer Science at University College London (UCL), advised by Prof. Sebastian Riedel and Prof. Pontus Stenetorp. He is going to join City University of Hong Kong as an Assistant Professor in Fall 2023. His research mainly focuses on Question Answering, Data Generation, and Knowledge-augmented Pre-trained Language Models. He has published 12 papers in top AI/NLP conferences and journals (ACL, EMNLP, AAAI, IJCAI, KDD, TACL). His works are cited by more than 1600 times, and won Best Paper Award AKBC2020, champion of Neurips 2020 EfficientQA Competition, and Best Poster Award ENLSP Workshop in NeurIPS 2022. Personal home page: https://jimmycode.github.io/ Add to your calendar vCal iCal Feb 07 2023 11.00 - 12.00 Friday, 10th February 2023 - 11am Yuxiang Wu : Seminar This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk. Informatics Forum, G.03 and online by invitation Contact
Feb 07 2023 11.00 - 12.00 Friday, 10th February 2023 - 11am Yuxiang Wu : Seminar This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk.
