# OpenCV: How to get the “rectified” fundamental matrix?

I have a stereo image pair and the respective intrinsics and extrinsics of both cameras. With this information, I can calculate the fundamental Matrix between the two cameras (let's call it F).

I can also use this information to rectify both images. In the rectified images I then find features, match them, and approximate the fundamental matrix. This approximated fundamental matrix, F_rect, is of course a completely different one from the F calculated above.

How can I get the "true" rectified fundamental matrix in OpenCV, so I can compare the approximated to the true one? My code looks something like this:

void rectify(cv::Mat& image1, cv::Mat& image2,
cv::Mat& RT1, cv::Mat& RT2,
cv::Mat& K1, cv::Mat& K2,
cv::Mat& dst_img1, cv::Mat& dst_img2,
cv::Mat* P1_new = nullptr, cv::Mat* P2_new = nullptr) {

cv::Mat RT(3,4, CV_64FC1);

LtoR(RT1, RT2, RT);
cv::Mat distortion(1,4, CV_64FC1);
cv::Mat R1, R2, P1, P2, Q;

cv::stereoRectify(K1, distortion, K2, distortion, image1.size(),
RT(cv::Rect(0,0,3,3)), RT(cv::Rect(3,0,1,3)), R1, R2, P1, P2, Q,
/*flags=*/0, /*alpha=*/1);
cv::Mat map1x, map1y, map2x, map2y;
cv::initUndistortRectifyMap(K1, distortion, R1, P1, image1.size(), CV_32FC1, map1x, map1y);
cv::initUndistortRectifyMap(K2, distortion, R2, P2, image2.size(), CV_32FC1, map2x, map2y);

cv::remap(image1, dst_img1, map1x, map1y, cv::INTER_LINEAR);
cv::remap(image2, dst_img2, map2x, map2y, cv::INTER_LINEAR);

if (P1_new != nullptr) {
*P1_new = P1.clone();
}
if (P2_new != nullptr) {
*P2_new = P2.clone();
}
}


stereoRectify() returns the rotation matrices to rotate the cameras into the rectified coordinate systems and the new projection matrices.

initUndistortRectifyMap returns a transformation, so that the images are rectified after this transformation.

I think I would need the rectifying homography (maybe?), these functions don't return it. Any other way?

TL;DR I rectify a stereo image pair. I have the true fundamental matrix before rectification, and an approximated fundamental after rectification. How do I get the true fundamental after rectification?

## migrated from cs.stackexchange.comFeb 21 at 12:04

This question came from our site for students, researchers and practitioners of computer science.