Learning Under Constraints Inference, Selection and Validation with Maximum Entropy Models

Maximum entropy models are probabilistic models that reproduce a prescribed set of empirical constraints while leaving everything else maximally random. They provide a general framework for con- structing reference models, rooted in information theory, and are particularly useful for studying complex systems. This thesis investi- gates the use of maximum entropy models in statistical inference, and in particular for model selection, hypothesis testing, and statistical validation. The first chapter introduces Maximum Entropy Models and the main statistical inference problems addressed throughout the thesis. A focus is put on the distinction between canonical models, which reproduce the constraints on average, and microcanonical models, which reproduce the constraints exactly. These two formulations are known to become inequivalent when an extensive number of constraints are enforced. The second chapter of the thesis studies how to choose between these two formulations with model section. Using the Minimum Description Length principle, canonical and microcanonical models are compared by balancing their fit to the data against their complex- ity. The results show that the microcanonical formulation provides a better fit but is also more complex. The chapter shows that the balance between fit and complexity becomes relevant when the two formulations are inequivalent. The third and fourth chapters develop e-values for hypothesis testing between maximum entropy models. E-values provide a modern and flexible way to quantify statistical evidence. The proposed methods are first studied theoretically, providing analytical solutions to derive optimal e-values. Secondly, applications to contingency tables are introduced and studies in details, providing practical considerations for practitioners. The fifth chapter applies maximum entropy models as null models for validating patterns in brain activity. Resting-state fMRI data are represented as interactions between brain regions and time points, and observed pairwise and triadic co-activations are compared with those expected under the null models. This analysis reveals distinct forms of triadic coordination, including both spatially compact in- teractions supported by pairwise connections and more distributed interactions without such support.