Pixel-To-Angle Transformation in Camera Image

I'm trying to localize points I see in a camera image in terms of azimuth and elevation and match points between shots. Individual shots should differ only in rotation around the camera's center (roughly). It's not working (as I'll explain below), and I'm looking for things I might be doing wrong.

I've tried different things, but this is what I'm doing right now: Let's say I have tilted my camera (around the $$x$$-axis) by some angle $$\alpha$$ and rotated it (around the $$y$$-axis) by $$\beta$$. Assuming my lens gives me a perspective projection with a focal length of $$d$$, I can represent pixel $$(i,j)$$ as a vector $$\vec{v}=d\cdot\vec{z}' + i\cdot\vec{x}' + j\cdot\vec{y}',$$ where $$\vec{x}', \vec{y}', \vec{z}'$$ are the unit vectors of the standard basis rotated by $$\alpha$$ and $$\beta$$ around x-axis and y-axis, resp. $$\vec{v}$$ then represents the point in 3D space at which the photoreceptor cell in the CMOS was when it perceived the point in real space.

Say $$\vec{v}=(v_1,v_2,v_3)$$. Then I should be able to calculate azimuth $$\phi$$ and elevation $$\psi$$ wrt. an unrotated take as: $$\phi = \mathrm{arctan}(\frac{v_1}{v_3})\quad\text{and}\quad\psi = \mathrm{arcsin}(\frac{v_2}{|\vec{v}|}).$$

This doesn't work, though. That is, without tilting the camera it more or less does. But say I have a shot where both $$\alpha$$ and $$\beta$$ are not zero. Then, the points which appear in that shot and in others for which $$\alpha$$ or $$\beta$$ are zero don't come out with the same azimuth and elevation. It seems that they generally cluster too much around the center of that shot.

One of the problems is that there are a lot of things that could get in my way here, like any combination of:

• a distorted camera image
• wrong focal length
• wrong rotation angles
• rotation around a point too far from the focal point.

I'm looking for more potential sources of error and (in)validation for the approach outlined above.

Above is a visualization of the outcome of the procedure so far: the white horizontal and vertical axes are azimuth and elevation, resp. The distorted rectangles are boundaries of individual shots. It's hard to explain what exactly the points are, but in essence points with the same color are identical in the real world and should, therefore, appear at the same absolute azimuth and elevation. These points form a distorted shape, not a rectangle. You can see that at each level of elevation, the points from each camera image appear more and more contracted around the camera image's center.

• Could you rephrase this slightly so that it ends with a sentence containing a question mark? – Richard Aug 25 at 15:55