Friday, 23rd May - 12pm, Rob Procter Speaker: Professor Rob Procter (Department of Computer Science, University of Warwick and Alan Turning Institute)Title: Mind the Gap: Are NLP Benchmarks Fit-for-Purpose?Abstract: Benchmarks have evolved as a tool for NLP researchers to measure performance of a model on a specific task. Drawing on work with collaborators in the AdSoLve project, I will review evidence on whether existing benchmarking principles and practices remain effective in the context of advanced NLP technologies, such as LLMs, and their rapid adoption. I will then summarise results from a recent literature review of benchmarking LLM performance in legal use cases. I will conclude with some recommendations for ensuring benchmark techniques meet the requirements of law practitioners and service providers of ‘lawtech’. May 23 2025 12.00 - 13.00 Friday, 23rd May - 12pm, Rob Procter This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk. IF, G.03 and Teams
Friday, 23rd May - 12pm, Rob Procter Speaker: Professor Rob Procter (Department of Computer Science, University of Warwick and Alan Turning Institute)Title: Mind the Gap: Are NLP Benchmarks Fit-for-Purpose?Abstract: Benchmarks have evolved as a tool for NLP researchers to measure performance of a model on a specific task. Drawing on work with collaborators in the AdSoLve project, I will review evidence on whether existing benchmarking principles and practices remain effective in the context of advanced NLP technologies, such as LLMs, and their rapid adoption. I will then summarise results from a recent literature review of benchmarking LLM performance in legal use cases. I will conclude with some recommendations for ensuring benchmark techniques meet the requirements of law practitioners and service providers of ‘lawtech’. May 23 2025 12.00 - 13.00 Friday, 23rd May - 12pm, Rob Procter This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk. IF, G.03 and Teams
May 23 2025 12.00 - 13.00 Friday, 23rd May - 12pm, Rob Procter This event is co-organised by ILCC and by the UKRI Centre for Doctoral Training in Natural Language Processing, https://nlp-cdt.ac.uk.
