Pre-viva Talk - 02/10/2025 Speaker: Jay ParkTitle: Exploiting Generics During Interactive Task LearningAbstract: Recent machine learning (ML) research has chiefly focused on optimising performance on tasks whose structure is fully specified in advance. For instance, with assembly tasks, the agent has complete knowledge prior to deployment about which component parts form valid structures; e.g., that fire trucks must have ladders. The agent may not know, and will have to learn, what fire trucks or what ladders look like, but they have a complete and accurate hypothesis space of all the concepts that they will need. It is common to assume that the learning system has access to complete knowledge of this kind, but the assumption often fails in real-world applications. The learner may initially lack awareness of key concepts in the domain ontology and/or crucial relations among them, and the task domain may evolve in unforeseen ways after the agent is deployed. Interactive task learning (ITL) is a ML paradigm which addresses this challenge by enabling the learner to cope with ever-changing hypothesis spaces: the key insight in ITL is to use continued communication with (human) domain experts as evidence for learning to solve novel tasks.This thesis explores ITL in knowledge-intensive tasks grounded in low-level sensory input spaces. To support effective and efficient ITL in such domains, I devise a neurosymbolic system architecture in which instances of task-critical concepts are grounded by neural models and serve as a basis for symbolic reasoning. Existing neurosymbolic methods either posit full knowledge of symbolic domain ontologies in advance or rely on offline batch-learning. I demonstrate how we can relax these assumptions by virtue of teacher-learner interactions that occur while the agent is trying to solve its task. These interactions are in the form of embodied natural language dialogues featuring not only demonstratives (e.g.,``This is a ladder") but also generics (e.g., ``Dump trucks have yellow cabins"). Generics express generalised relations among concepts (e.g., similarity-difference, holo/meronymy, hyper/hyponymy), allowing the learner to incrementally integrate novel concepts into the domain ontology as and when required.I design and implement a suite of ITL frameworks that leverage generics to support the incremental construction of symbolic knowledge through natural dialogue with a human supervisor. The utility of generics in ITL is evaluated across a range of probing tasks, from fine-grained visual grounding to long-horizon planning. Empirical results show that agents capable of interpreting the semantics and pragmatics of generics acquire novel tasks more data-efficiently compared to ablative baselines. Oct 02 2025 13.00 - 14.00 Pre-viva Talk - 02/10/2025 Jay Park G.03
Pre-viva Talk - 02/10/2025 Speaker: Jay ParkTitle: Exploiting Generics During Interactive Task LearningAbstract: Recent machine learning (ML) research has chiefly focused on optimising performance on tasks whose structure is fully specified in advance. For instance, with assembly tasks, the agent has complete knowledge prior to deployment about which component parts form valid structures; e.g., that fire trucks must have ladders. The agent may not know, and will have to learn, what fire trucks or what ladders look like, but they have a complete and accurate hypothesis space of all the concepts that they will need. It is common to assume that the learning system has access to complete knowledge of this kind, but the assumption often fails in real-world applications. The learner may initially lack awareness of key concepts in the domain ontology and/or crucial relations among them, and the task domain may evolve in unforeseen ways after the agent is deployed. Interactive task learning (ITL) is a ML paradigm which addresses this challenge by enabling the learner to cope with ever-changing hypothesis spaces: the key insight in ITL is to use continued communication with (human) domain experts as evidence for learning to solve novel tasks.This thesis explores ITL in knowledge-intensive tasks grounded in low-level sensory input spaces. To support effective and efficient ITL in such domains, I devise a neurosymbolic system architecture in which instances of task-critical concepts are grounded by neural models and serve as a basis for symbolic reasoning. Existing neurosymbolic methods either posit full knowledge of symbolic domain ontologies in advance or rely on offline batch-learning. I demonstrate how we can relax these assumptions by virtue of teacher-learner interactions that occur while the agent is trying to solve its task. These interactions are in the form of embodied natural language dialogues featuring not only demonstratives (e.g.,``This is a ladder") but also generics (e.g., ``Dump trucks have yellow cabins"). Generics express generalised relations among concepts (e.g., similarity-difference, holo/meronymy, hyper/hyponymy), allowing the learner to incrementally integrate novel concepts into the domain ontology as and when required.I design and implement a suite of ITL frameworks that leverage generics to support the incremental construction of symbolic knowledge through natural dialogue with a human supervisor. The utility of generics in ITL is evaluated across a range of probing tasks, from fine-grained visual grounding to long-horizon planning. Empirical results show that agents capable of interpreting the semantics and pragmatics of generics acquire novel tasks more data-efficiently compared to ablative baselines. Oct 02 2025 13.00 - 14.00 Pre-viva Talk - 02/10/2025 Jay Park G.03
