Title

Grounded language learning, from sounds and images to meaning

Bio
I am an Associate Professor at the Department of Cognitive Science and Artificial Intelligence at Tilburg University, the Netherlands. I am mainly interested in studying language learning using computational models. My main area of research is modeling grounded language learning, the impact of interaction in language acquisition and processing, and developing tools and techniques for better understanding of emergent linguistic representations in (neural) models of language.

Abstract

Humans learn to understand speech from weak and noisy supervision: they extract linguistic structure and meaning from speech by simply being exposed to utterances situated and grounded in their daily sensory experience. Emulating this remarkable skill has been the goal of numerous studies; however researchers have often used severely simplified settings where either the language input or the extralinguistic sensory input, or both, are small-scale and symbolically represented. I present a series of studies on modelling visually grounded language understanding. Using various deep architectures to model the temporal nature of spoken language, we examine how form and meaning-based linguistic knowledge emerges from the input signal, and how we can identify and analyze this knowledge.