Goals of the chair
The project has a high degree of continuity with the great success of the 3IA Aniti and DEEL program: studying 1-Lipschitz neural networks, a class of robust by-design neural networks. The team, built in this first round, was composed of Mathematicians, Computer Science researchers, Data-scientists and Industry experts, and has published several papers in major conferences and journals. They have laid the groundwork for classification with 1-Lipschitz neural networks, both on theory, on the definition of optimal loss linked to optimal transport, and proof of robustness, certifiability and explainability. Additionally, a full library has been developed, called DEEL-LIP , to learn these kind of neural networks as classical Tensorflow or PyTorch models.
Principal investigators
- Mathieu Serrurier (Professor, UT2, IRIT ADRIA-DEEL)
- Franck Mamalet (Senior Expert AI, IRT Saint Exupéry, DEEL)
In this project, we propose to further investigate the 1-Lipschitz neural networks in the scope of self-supervised learning: the objective is to be able to learn large models with unannotated data in several domains (medical/satellite images, time series, natural language processing) while maintaining the guarantees in terms of robustness, certifiability and explainability. Self-supervised learning is a high trend for classical networks with applications in few-shot learning, semi-supervised learning or as backbones. But, as far as we know, there is no contribution in the literature on self-supervised 1-Lipschitz neural networks.
Co-chairs
-
- Mélanie Ducoffe (Research Engineer, AIRBUS)
- Coralie Sengenès (CR INSERM, RESTORE)
We propose to tackle the unexplored domain of self-supervised 1-Lipschitz large models with three research axis: In the first axis, we will explore methods for self-supervised learning using optimal transport loss, to learn from unannotated data while still promoting the robustness of the neural network.
We will also investigate more recent and deeper 1-Lipschitz architectures, such as transformer, to enhance the learning capabilities of these networks on very large datasets and their generalisation. We will work to establish the theory and certifiable guarantees for these self-supervised learnt 1-Lipschitz Neural Networks.
To end with, we will evaluate the complementarity between 1-Lipschitz guarantees and formal methods, by extending the open-source library DECOMON.
For industrial safety-critical applications, we will develop a set of pre-trained 1-Lipschitz Networks for various domains, including satellite images, time series, language processing, and medical imaging, where data quantity and annotation are crucial. Industrial partners from the aerospace automotive and railway industries (Airbus, SNCF, Thales, Renault) will second engineers or CIFRE PhD and provide challenging use cases where robust methods are expected. This chair will also benefit from the framework of the collaborative integrative program DEEL of the ANITI AI Cluster, jointly designed and operated with IRT Saint Exupéry in collaboration with the project's academic and industrial partners. We will also investigate the usage of 1-Lipschitz NN in the health sector with the support of the RESTORE institute from Inserm. Mélanie Ducoffe (Research Expert, Airbus) and Coralie Sengenes (Inserm, RESTORE) are co-chairs.
- Extension of the DEEL-LIP library including new 1-Lipschitz layers and architectures, as well as pipelines for self-supervised learning, semi-supervised learning, and fine-tuning.
- Large open-source unannotated datasets for multimodal fluorescence microscopy images, satellite images, and NLP for air transportation, created by aggregating various open-source datasets.
- Distillation and fine-tuning with transferable guarantees of robustness, explainability, and fairness.
- Large-scale 1-Lipschitz network architectures for vision and natural language processing (NLP).
- Self-supervised learning algorithms for 1-Lipschitz networks on large unannotated datasets, based on optimal transport approaches.
- Distillation, fine-tuning, and few-shot learning approaches with transferable robustness, explainability, and fairness properties for 1-Lipschitz networks.
English 
French