Participant 1 : Alexandre Bonlarron
Titre : GenCP: Merging Intuition with Reasoning for Constrained Text Generation
Date : Le 20/03/2025 à 11h
Résumé :Human cognition seamlessly blends fast, instinctive decision-making (System 1) with slow, deliberate reasoning (System 2). In AI, Large Language Models (LLMs) excel at fluency but struggle with strict constraints, while Constraint Programming (CP) enforces rules but lacks the ability to capture such ill-posed constraints as fluency. This talk introduces GenCP [1], a novel approach that tightly integrates these two paradigms for constrained text generation. Unlike traditional hybrid approaches where CP and LLMs operate in a loosely coupled fashion [2,3,4] , GenCP embeds LLMs directly into the search process, dynamically guiding generation while ensuring strict adherence to constraints. We illustrate this evolution from ranking-based methods to fully interactive (i.e., LLM alongside GenCP), constraint-driven text generation. Experiments on tasks from the COLLIE benchmark [5] show that GenCP outperforms classical decoding strategies, producing more reliable and efficient results.
[1] Florian Régin, Elisabetta De Maria, and Alexandre Bonlarron. ‘Combining Constraint Programming Reasoning with Large Language Model Predictions’. In: 30th International Conference on Principles and Practice of Constraint Programming (CP 2024). 2024.
[2] Alexandre Bonlarron and Jean-Charles Régin. ‘Markov Constraint as Large Language Model Surrogate’. In: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24. Ed. by Kate Larson. Main Track. International Joint Conferences on Artificial Intelligence Organization, Aug. 2024, pp. 1844–1852. doi: 10.24963/ijcai.2024/204
[3] Alexandre Bonlarron and Jean-Charles Régin. ‘Intertwining CP and NLP: The Generation of Unreasonably Constrained Sentences’. In: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24. Ed. by Kate Larson. AI, Arts & Creativity. International Joint Conferences on Artificial Intelligence Organization, Aug. 2024, pp. 7600–7608. doi: 10.24963/ijcai.2024/841
[4] Alexandre Bonlarron et al. ‘Constraints First: A New MDD-based Model to Generate Sentences Under Constraints’. In: Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23. Ed. by Edith Elkind. 2023, pp. 1893–1901. doi: 10.24963/ijcai.2023/210
[5] Shunyu Yao, Howard Chen, Austin W. Hanjie, Runzhe Yang, and Karthik R Narasimhan. COLLIE: Systematic construction of constrained text generation tasks. In The Twelfth International Conference on Learning Representations, 2024.
