I have a stereo image pair and the respective intrinsics and extrinsics of both cameras. With this information, I can calculate the fundamental Matrix between the two cameras (let's call it F).

I can also use this information to rectify both images. In the rectified images I then find features, match them, and approximate the fundamental matrix. This approximated fundamental matrix, F_rect, is of course a completely different one from the F calculated above.

How can I get the "true" rectified fundamental matrix in OpenCV, so I can compare the approximated to the true one? My code looks something like this:

void rectify(cv::Mat& image1, cv::Mat& image2,
            cv::Mat& RT1, cv::Mat& RT2,
            cv::Mat& K1, cv::Mat& K2,
            cv::Mat& dst_img1, cv::Mat& dst_img2,
            cv::Mat* P1_new = nullptr, cv::Mat* P2_new = nullptr) {

    cv::Mat RT(3,4, CV_64FC1);

    LtoR(RT1, RT2, RT);
    cv::Mat distortion(1,4, CV_64FC1);
    cv::Mat R1, R2, P1, P2, Q;

    cv::stereoRectify(K1, distortion, K2, distortion, image1.size(),
                        RT(cv::Rect(0,0,3,3)), RT(cv::Rect(3,0,1,3)), R1, R2, P1, P2, Q,
                        /*flags=*/0, /*alpha=*/1);
    cv::Mat map1x, map1y, map2x, map2y;
    cv::initUndistortRectifyMap(K1, distortion, R1, P1, image1.size(), CV_32FC1, map1x, map1y);
    cv::initUndistortRectifyMap(K2, distortion, R2, P2, image2.size(), CV_32FC1, map2x, map2y);

    cv::remap(image1, dst_img1, map1x, map1y, cv::INTER_LINEAR);
    cv::remap(image2, dst_img2, map2x, map2y, cv::INTER_LINEAR);

    if (P1_new != nullptr) {
        *P1_new = P1.clone();
    if (P2_new != nullptr) {
        *P2_new = P2.clone();

stereoRectify() returns the rotation matrices to rotate the cameras into the rectified coordinate systems and the new projection matrices.

initUndistortRectifyMap returns a transformation, so that the images are rectified after this transformation.

I think I would need the rectifying homography (maybe?), these functions don't return it. Any other way?

TL;DR I rectify a stereo image pair. I have the true fundamental matrix before rectification, and an approximated fundamental after rectification. How do I get the true fundamental after rectification?


migrated from cs.stackexchange.com Feb 21 at 12:04

This question came from our site for students, researchers and practitioners of computer science.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy