Zoom-Meeting: https://wwu.zoom.us/j/94525063967 If you are interested please send an email to martin.huesmann@uni-muenster.de
Stochastic Gradient Descent (SGD) is a powerful and widely used tool in the approximation of extrema of a function F, where, at each point x, we are only able to simulate a noisy version of the gradient f = DF(x). Nowadays, a key application is the training of neural networks. Most estimates on the rate of convergence of SGD schemes have only been shown under global convexity assumptions on the objective function. However, it is well known that the loss-function of a typical machine learning problem is non-convex and possibly contains multiple global minima. We introduce a second update scheme that explores the state space and helps SGD find the neighbourhood of a global minimum. Afterwards, we give sufficient conditions on both schemes and their interference to guarantee convergence to one of the global minima.