Column-normalized inverse?

Question

Suppose we define $A^{*}$ of positive definite $A=X'X$ using following two steps:

1. let $B=A^{-1}$
2. scale columns of $B$ to obtain a matrix with $1$'s on the diagonal

For the case of singular $A$, we could use a tiny amount of Tikhonov regularization: $A^*=\lim_{\epsilon\to 0}(A+\epsilon I)^*$

Is there a name of this operation, or a way to compute it efficiently? Regularized inverse is efficient, but what to use for $\epsilon$?

Replacing regularized inverse with pseudo-inverse in step 1. gives a dramatically different result.

For instance, if we define $A=X'X$ with

$$X=\left( \begin{array}{ccc} 2 & -2 & 2 \\ -1 & 1 & 1 \\ -2 & -2 & 1 \\ 1 & 1 & 3 \\ \end{array} \right)$$

Then $(X'X)^{*}$ seems to be

$$\left( \begin{array}{cccc} 1 & \frac{1}{2} & -\frac{7}{4} & -\frac{7}{8} \\ 2 & 1 & -\frac{7}{2} & -\frac{7}{4} \\ -\frac{4}{7} & -\frac{2}{7} & 1 & \frac{1}{2} \\ -\frac{8}{7} & -\frac{4}{7} & 2 & 1 \\ \end{array} \right)$$

Motivation: empirically this operation seems give a way to obtain least-squares fitted coefficients of an autoregressive model of x, $x\approx Bx$ with diagonal of $B$ restricted to be 0's: $$B=I-(X'X)^*$$

Checks in colab

Edited with answer

The formula for non-singular version

$$B=(X'X)^{-1} (\text{zero_off_diagonal}[(X'X)^{-1}])^{-1}$$

where "zero_off_diagonal" sets all off-diagonal entries to zero.

Formula for singular version

$$P=I-X^\dagger X\\ B=P(\text{zero_off_diagonal}(P))^{-1}$$

notebook

Answer 1

$ \def\m#1{\left[\begin{array}{r}#1\end{array}\right]} \def\e{\epsilon} \def\qiq{\quad\implies\quad} \def\LR#1{\left(#1\right)} \def\Diag#1{\operatorname{Diag}\left(#1\right)} $Construct the orthoprojector $\LR{P^T=P=P^2}$ for the nullspace of $X$ $$\eqalign{ P &= {I-X^+X} \qiq XP=0 \qquad\qquad\quad \\ }$$ and use it to construct the function $$\eqalign{ D &= \Diag{P} \qiq \LR{X^TX}^* &= P D^{-1} \\ }$$ Calculating a pseudoinverse is more reliable than approximating a limit with an arbitrary $\e$.

Although there is a limit expression for the pseudoinverse, it's numerically unstable $$\eqalign{ X^+ = \lim_{\e\to 0}\LR{X^TX+\e I}^{-1}X^T \\\\ }$$

---

In terms of the motivating model, note that $$\eqalign{ B \;=\; I-\LR{X^TX}^* \;=\; \LR{D-P} D^{-1} \\ }$$

Update

Exact (rational) calculations for the pseudoinverse and the orthoprojector yield $$\eqalign{ X^+ &= \frac{1}{650}\m{ 114 & -122 & 40 \\ -97 & 81 & 80 \\ -158 & -116 & 70 \\ 9 & 93 & 140} \\ P &= \frac{1}{325}\m{ 49 & 98 & -28 & -56 \\ 98 & 196 & -56 & -112 \\ -28 & -56 & 16 & 32 \\ -56 & -112 & 32 & 64 } }$$ resulting in $$\eqalign{ D^{-1} &= \m{ \frac{325}{49} & 0 & 0 & 0 \\ 0 & \frac{325}{196} & 0 & 0 \\ 0 & 0 & \frac{325}{16} & 0 \\ 0 & 0 & 0 & \frac{325}{64} } \\ PD^{-1} &= \m{ 1 & \frac{1}{2} & -\frac{7}{4} & -\frac{7}{8} \\ 2 & 1 & -\frac{7}{2} & -\frac{7}{4} \\ -\frac{4}{7} & -\frac{2}{7} & 1 & \frac{1}{2} \\ -\frac{8}{7} & -\frac{4}{7} & 2 & 1 } }$$