What would then be the way to determine a true dimensionality of a configuration of points $X\in\mathbb{R}^{n\times k}$ based on its Gram matrix $G=XX^T$? The "true" dimensionality refers to the minimum number of dimensions needed to represent a configuration (eg, a line is 1D, but can be represented in 2D).
Different shifts would imply different ranks of $G$ that correspond to different dimensionalities. Consider, for instance, the paper http://convexoptimization.com/TOOLS/Gower1.pdf (Sect. 3, very short, please consider)
If I understand correctly, as long as the shifts are of the form $X'=PX$, $P=I_n-1_nw^T$, where $1_n^Tw=1$, the rank of $G'=X'(X')^T$ is the true dimensionality of $X$? If you consider the 2x2 example from the above paper, a shift where one point is the origin reduces the dimensionality to 1D, which is the true dimensionality (same holds for a centroid origin) Does that mean that, in order to recover the "true" configuration, one is restricted to the above form of the shift?