Friday 10 October 2025 - 11am

Speaker: Adi Simhi (Technion, Israel Institute of Technology)

Title: Characterizing Hallucinations: A Two-Dimensional Analysis of Knowledge and Certainty

Abstract: Hallucinations in Large Language Models (LLMs) present challenges beyond simple knowledge gaps. This work investigates hallucinations that occur despite models possessing correct information, examining two key studies: Distinguishing Ignorance from Error in LLM Hallucinations and Trust Me, I'm Wrong: High-Certainty Hallucinations in LLMs.

We address whether hallucinations can manifest when models know correct answers, developing methodologies to isolate hallucination-despite-knowledge cases from ignorance-based errors. This distinction proves crucial for mitigation. We then investigate whether hallucinations despite knowledge can occur with high certainty, finding that such high-certainty hallucinations exist and occur consistently. These findings challenge current uncertainty-based detection and mitigation methods. Lastly, we provide a novel way to evaluate mitigation methods.

Biography: Adi Simhi is a third-year PhD student in Yonatan Belinkov's lab at the Technion. Her research focuses on hallucinations and safety in LLMs. She received her Master's degree from the Technion under Shaul Markovitch.