A new paper on the geometry of the loss function in deep net using - TopicsExpress



          

A new paper on the geometry of the loss function in deep net using the methods of random matrix theory. arxiv.org/abs/1412.0233 The Loss Surface of Multilayer Networks by Anna Choromanska, Mikael Henaff, Michael Mathieu, Gérard Ben Arous, Yann LeCun. Bottom line: yes, deep nets have lots of local minima, but they are all more of less equivalent. It doesnt really matter which one you fall into. There are a few high-energy minima, but their number is tiny compared to the ones in the so-called barrier where most of them reside. Also, deep nets have lots and lots of saddle points (a combinatorially large number of them). So, optimizing a deep net is really a game of not getting too close to saddle points.
Posted on: Tue, 23 Dec 2014 23:12:47 +0000

Trending Topics



Recently Viewed Topics




© 2015