Suppose $ q $ and $ A $ are given and that $ q, p \in R^{N} $ and $ A \in R^{NxN} $, then how can I find the vector $ p $ such that $$ (q - p)^{T}A(q - p) $$ is minimized constraint to $ \sum_{i=1}^{N} |p_{i}| \leq 1 $ (thus the $ ||p||_{1} \leq 1 $ ) using projected gradient descent?
Is it also possible to apply projected gradient descent with the constraint of $ ||p||_{1} = 1 $ as well?
I can find the gradient of the cost function itself, but given the "updated" $p$ value, I need to project it via $argmin_{p^{*}} (||p - p^{*}||)$ where the $p^{*}$, which is what I want, is the updated p value projected into the feasible space, but I'm not sure how I can go about doing that part.
g(x)=0
, you can just solve the rootfinding problem using something like Newton's method to project to the manifold. If your steps aren't large (i.e. the learning rate isn't too big) then each step will be close to the manifold and the projection won't be much of a change. $\endgroup$ – Chris Rackauckas Oct 27 '17 at 4:30