replace $A$ by a factored form from which the determinant can be computed stably and cheaply. E.g., add new triangular variables $L$ and $R$ and the constraint $A=PLR$ for some fixed permutation matrix $P$. Then determinants and their gradients are trivially to compute.

In case the solution has no such factorization, you need to change $P$ after a preliminary round of minimization with $P=I$.

Replace $A$ by a factored form from which the determinant can be computed stably and cheaply. E.g., add new triangular variables $L$ and $R$ and the constraint $A=PLR$ for some fixed permutation matrix $P$; often the identity permutation $P=I$ is enough. This means that you treat the nontrivial entries of $L$ and $R$ as additional variables. Then determinants and their gradients are trivially to compute. If $A$ is known to be symmetric, you may use instead $A=LDL^T$.

Alternatively you may eliminate $A$ by substituting $PLR$ for it everywhere it occurs. Thus in place of minimizing $f(A,\det A)$ subject to $F(A)=0$, say, you would minimize $f(LR,\det R)$ subject to $F(LR)=0$, (assuming for simplicity that $L$ is unit lower triangular and no permutation was applied), and use $\det R=\prod R_{ii}$.

In case the solution has no such factorization, you need to change $P$ after a preliminary round of minimization with $P=I$.