# How to choose a good step size for stochastic gradient descent?

For the purpose of model fitting in a large time series dataset, I am using stochastic gradient descent of the negative log likelihood. The model is nonlinear and non-convex. Is there a thumb rule for choosing a good step size? I could choose a very small step size for stable but painfully slow convergence, but I would like to be able to choose a big enough step size for faster convergence and anneal it.

The Netflix competition is a great example of a huge dataset (480,189 users and 17,770 movies, and several million ratings for the training set) where stochastic gradient descent was used as the workhorse optimization algorithm for training most of the prediction models used. A good paper to read regarding an algorithm used in the Netflix competition is Factorization Meets the Neighborhood by one of the winners of the competition, Y. Koren. For training their models, they typically used a fixed learning rate, and empirically a learning rate $\eta = 0.001$ seemed to work well for the Netflix problem. This is highly application specific, however!