Towards demystifying over-parameterization in deep (...) - Soltanolkotabi - Workshop 3 - CEB T1 2019

Mahdi Soltanolkotabi (USC) / Towards demystifying over-parameterization in deep learning. Many modern learning models including deep neural networks are trained in an over-parameterized regime where the parameters of the model exceed the size of the training dataset. Training these models involve highly non-convex landscapes and it is not clear how methods such as (stochastic) gradient descent provably find globally optimal models. Furthermore, due to their over-parameterized nature these neur