I want to minimize the following with respect to parameters $B$.

$$\sum_{k = 1}^{K} f(A_{k}, B)$$

where $A_k$ are $K$ different data-sets and $B$ is a matrix of parameters.

Can I do this by a gradient descent where:

  1. randomly pick one matrix $A_{i}$ among $K$ matrices.
  2. perform a gradient step that minimizes $f(A_{i}, B)$
  3. repeat steps 1 and 2 until convergence.

The function $f$ is the logarithm of a linear function.

After searching online, this looks like a stochastic gradient descent. Is this really an SGD?

  • $\begingroup$ Yes this is SGD. $\endgroup$ – Nick Alger Jan 15 at 21:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.