Single point feature matching

Image matching is often involved in many fields. From the early image matching based on gray value to today's feature matching, it can be said that image matching plays a very important role in the fields of computer vision, photogrammetry, remote sensing and so on. Feature matching generally includes three steps: feature extraction, feature description and feature matching. Today, we mainly discuss the related algorithms of feature extraction and feature description.

There are many excellent algorithms to deal with the related problems of feature extraction and description. The most representative is SIFT algorithm, which has good robustness, which is also the algorithm we want to discuss today. Of course, in addition to sift algorithm, there are many other excellent algorithms, such as SURF, ASIFT, BRISK, ORB and so on, which have their own characteristics.

Today, we mainly use SIFT algorithm in opencv library to realize single point feature matching in c + + environment. In daily application, we can directly call sift related functions to realize feature extraction and description. The code is as follows:

  • SIFT feature extraction:
CV_WRAP virtual void detect( InputArray image,
                             CV_OUT std::vector<KeyPoint>& keypoints,
                             InputArray mask=noArray() );
//images is the input image
//keypoints is the sequence of key points
//Mask is a mask that specifies the area for detecting key points, which can be ignored in general
  • SIFT Feature Description:
    CV_WRAP virtual void compute( InputArray image,
                                  CV_OUT CV_IN_OUT std::vector<KeyPoint>& keypoints,
                                  OutputArray descriptors );
//images is the input image
//keypoints is the sequence of key points
//Descriptors are subsequences of descriptors
  • Feature extraction and description examples
cv::Ptr<cv::Feature2D> f2d = cv::SIFT::create();//Instantiate SIFT
std::vector<cv::KeyPoint> kpts;//Key sequence
cv::Mat desc;//Definition descriptor
cv::Mat img = cv::imread("Your filename");//Read image
f2d->detect(img, kpts);//Calculate key kpts
f2d->compute(img, kpts, desc);//Calculation descriptor desc

You can also directly extract key points and calculate descriptors

f2d->detectAndCompute(img, cv::Mat(), kpts,desc);

The above is the process of feature extraction and description in general. The program will automatically extract points with strong features as feature points according to the image, and then describe them for further image matching. However, in some places with special needs, such as for two images, we need to match a specified point on the left image with the right image to obtain a unique matching point pair. In this case, the program cannot extract points with strong features as feature points, because at this time, the position of feature points has been specified for the left image. Take a look at the structure of the KeyPoint type:

KeyPoint(Point2f _pt, float _size, float _angle=-1, float _response=0, int _octave=0, int _class_id=-1);
//_ x, y coordinates of pt key
//_ size SIFT scale parameter
//_ angle key direction parameter
//_ Response key point detector response parameters to key points
//_ octave the number of layers of the detected key points in the image pyramid
//_ class_id number

As can be seen from the above structure, if you want to specify a specific point as a key point, you only need to_ Specify the x and y coordinates in the pt parameter, and set a scale parameter of SIFT at the same time_ Size is enough. In general_ The effect is better when the value of size is set to 3-5.

  • Examples of single point feature matching
cv::Ptr<cv::Feature2D> f2d = cv::SIFT::create();//Instantiate SIFT
std::vector<cv::KeyPoint> kpts1, kpts2;//Key sequence
cv::Mat desc1, desc2;//Definition descriptor
std::vector<cv::DMatch> matches;//Define matching point pairs
int SIFT_Size = 5;//SIFT scale parameters (3-5)
cv::Mat img1 = cv::imread("Your filename1");//Image reading 1
cv::Mat img2 = cv::imread("Your filename2");//Read image 2

cv::Point2f pt(231, 321);//Specify the coordinates of image 1 feature points
cv::KeyPoint kpt(pt, SIFT_Size);//Convert to feature points
kpts1.push_back(kpt);
f2d->compute(img1, kpts1, desc1);//Calculation descriptor desc1

//Traverse each pixel of image 2 and set it as a feature point
for (int i = 0; i < img2.rows; i++)
    for (int j = 0; j < img2.cols; j++)
        {
            cv::Point2f pt(i, j);
            cv::KeyPoint kpt(pt, 5);
            kpts2.push_back(kpt);
        }
f2d->compute(img2, kpts2, desc2);//Calculate the descriptors of all key points of image 2

//Match the specified feature points of image 1 with all feature points of image 2 to obtain the best matching point pair
cv::FlannBasedMatcher matcher;
matcher.match(desc1, desc2, matches);
cv::Mat outimg;
cv::drawMatches(img1, kpts1, img2, kpts2, matches, outimg, cv::Scalar(0, 255, 0), cv::Scalar(0, 255, 0), std::vector<char>(), cv::DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS);
cv::imshow("SIFT", outimg);
cv::waitKey(0);

The above is the relevant implementation code of single point feature matching, but it also has some defects:

  1. When setting the parameters of key points, all parameters except SIFT scale parameters are the default parameters. In fact, their parameters are different for different key points, which will have a certain impact on the matching results. Of course, you can look at the source code of SIFT and calculate these parameters together in the program calculation. Due to the limited level of the author, there will be no more explanation here;
  2. When traversing the pixels of image 2, each point is actually set as a key point, which has a large amount of computation in calculating the descriptor and matching, and has some impact on the speed of the program;
  3. For the specified point, sometimes its characteristics are not very obvious, so it will affect the accuracy of matching, which has high requirements for the descriptor; You can also try different descriptions.

Finally, an image effect picture generated when I did the single point matching experiment is attached.

Tags: C++ OpenCV

Posted by Waxxy on Mon, 02 May 2022 07:35:02 +0300