Chair objectives
Recent AI models have achieved remarkable success in specific domains (e.g. vision, language, robotic agent control), and there is a push towards ever larger models combining multiple input and output modalities.
Inctheory, multimodal representations can help vision scientists by endowing sensory inputs with semantic information;
similarly, linguists can use them to ground NLP tokens in the sensorimotor environment and create a form of referential meaning;
Roboticists can also take advantage of these versatile representations for navigation and action planning.
Principal investigator
- Rufin VanRullen (Lead PI, DR CNRS, CerCo)
- Nicholas Asher (DR CNRS Emerite, IRIT)
- Thomas Serre (Professor, Brown University)
- Olivier Stasse (DR CNRS, LAAS)
But in practice, current models rely on brute-force training approaches using billions of labelled examples, while the datasets and computing resources available to academic and industrial researchers are typically much smaller.
Compared to artificial neural networks, real brains learn much more efficiently. We thus take inspiration from the cognitive science idea of a Global Workspace (GW) to build a
novel class of AI systems.
The GW, a unique model of multimodal grounding (encompassing perception, action and semantic representations), can promote advances in perceptual models and support both top-down interactions (from language and semantics to perception and action) of interest to
linguists and bottom-up interactions (from active perception and navigation to semantic abstractions) of interest to roboticists
The high-risk/high-gain hypothesis is that the modalities complement one another synergistically, such that the whole system is much more efficient than the sum of its
parts, not just for multimodal tasks but also when evaluated in the initial domains (vision, NLP, robotics).
Building frugal perceptual and cognitive models that can support language grounding and embodiment and provide semantic representations to robotic agents is expected to have important beneficial consequences for ANITI’s industrial partners (e.g. Airbus, Linagora).
Co-PI
-
- Alexandre Arnold (senior researcher, Airbus AI research accompanied by other Airbus personnel: Nahum Alvarez, Thiziri Belkacem, Nawal Ould-Amer and Caroline Sauget)
- Victor Boutin (researcher, CNRS, Serco)
- Thomas Flayols (Researcher, LAAS/CNRS)
- Julie Hunter (senior researcher, Linagora)
- Nicolas Mansard (Research Director, LAAS/CNRS)
- Philippe Muller (Associate Professor, UT3- IRIT: linguistics, NLP, deep learning)
- Thomas Pellegrini (Associate Professor, UT3-IRIT)
- Chloé Braud (Reseracher, CNRS, IRIT)
- Brain-inspired, frugal multimodal systems based on GW theory (PI: VanRullen)
- Perception with and without a GW (PI: Serre)
- Top-down track: from language semantics to situated conversational agents via GW grounding (PI:Asher)
- Bottom-up track: from mobile robots to situated conversational agents via GW grounding (PI: Stasse)
- TUM, ICS Technical University of Munich, Institute of Cognitive Sciences
- University of Colorado @ Boulder, University of London on Minecraft
- University of Potsdam on COCOBOT project
- Imperial College London (UK) and Araya Inc (Japan) on ERC Glow
- Brown University, George Karniadakis via T.Serre
- Airbus Defense and Space: satellite image super-resolution (Computer Vision team: Mathieu LeGoff)
- Labsoft
- improved multimodal systems (more frugal, robust, grounded, explainable…) and novel neuroscienceinspired learning architectures
- novel explainability methods (for e.g. computer vision)
- better models of semantics at the clausal and discourse level for NLP
- transition to Industry 4.0, i.e., from hard-coded robots to more collaborative ones
- cobots that can reliably accomplish complex tasks on an industrial shop floor
- robots that are able to understand simple orders in an industrial context
- cobots endowed with advanced conversational and interactional skills for many safety-critical tasks in various areas (airport traffic management, single pilot operations, elderly care)