Reach Out

📍Paris, France

🔬University Paris Saclay (MICS) - @CentraleSupelec

🎓 Scholar

🤗 HuggingFace

😺 GitHub

🐦 Twitter

💼 Linkedin

My Research 🔬

Recent advances in deep learning have considerably improved natural language processing (NLP) with Transformer architectures. However, their adoption remains a challenge in terms of training costs, deployment and evaluation for real-life commercial applications.

My current thesis work focuses therefore on :

• Adaptation: Optimising LLM Adaptation Strategies for Specific Distributions.

• Compression: How to improve performance while reducing model complexity.

• Evaluation: Performance evaluation of compressed models with consistent metrics for commercial applications.

News 📰

🧪 Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs

19/02/2024

Deploying large language models (LLMs) of several billion parameters can be impractical in most industrial use cases due to constraints such as cost, latency limitations, and hardware accessibility. Knowledge distillation (KD) offers a solution by compressing knowledge from resource-intensive large models to smaller ones. In this paper, we introduce Universal Logit Distillation (ULD) loss. ULD loss enabling distillation across models with different architectures and tokenizers, paving the way to a more widespread use of distillation techniques.

🥐 CroissantLLM: A Truly Bilingual French-English Language Model

01/02/2024

We introduce CroissantLLM, a 1.3B language model pretrained on a set of 3T English and French tokens, to bring to the research and industrial community a high-performance, fully open-sourced bilingual model that runs swiftly on consumer-grade local hardware.

Engineering Degree: JUNIA - ISEN Lille

01/09/2023

Graduate engineer ISEN Lille with a specialisation in artificial intelligence. A sincere thank you to all the ISEN teachers and researchers for these 5 years of enriching teaching.

🎥 ICASSP Paper : Deep Learning-Based Stereo Camera Multi-Video Synchronization

05/05/2023

Stereo vision is essential for many applications. Currently, the synchronization of the streams coming from two cameras is done using mostly hardware. A software-based synchronization method would reduce the cost, weight and size of the entire system and allow for more flexibility when building such systems. With this goal in mind, we present here a comparison of different deep learning-based systems and prove that some are efficient and generalizable enough for such a task. This study paves the way to a production ready software-based video synchronization system.