A constrained-based approach to machine learning

This paper promotes a constrained-based approach to machine learning as a natural evolution to classic distinction between supervised, unsupervised and semi-supervised learning. In particular, in addition to the unification of symbolic and sub-symbolic processing, it is shown that the notion of constraint opens the doors to a truly new way of discovering the solution that relies on the Lagrangian framework. In order to capture the idea, the paper focuses on the reformulation of supervised learning, thus proposing an algorithm that goes beyond the arguable Backpropagation biological plausibility. A biologically plausible local algorithm is proposed that is based on the search for saddle points in the learning adjoint space composed of weights, neural outputs, and Lagrangian multipliers. This might open the doors to a truly novel class of learning algorithms where, because of the introduction of the notion of support neurons, the optimization scheme also plays a fundamental role in the construction of the architecture.