CSVA

CSVA

Briefly

CSVA (“core” structural verification algorithm) is a robust and fast algorithm for outlier elimination of keypoint matches (e.g. SURF, SIFT or BRISK features) that heavily exploits gemetrical constraints. It can be used by developers and researches in different practical tasks with minor adaptation. Here is a summary for 3D scene matching task:

Method	matches in avg	precision, %
epipolar geometry	285	53
proprietry SIFT	55	79
CSVA	92	83

Examples

Below there are several applications developed with the use of Opencv feature extraction methods and CVSA.

3D image matching

By default CSVA runs in “3D mode”. It is robust to huge number of outliers.

Matches generated by SURF contains less than 6% of inliers

After CSVA set of matches contains 96% of inliers.

Registering aerospace images

If CSVA is lauchced in “flat” mode it can use more rigid hyperparameters. That makes possible registering aerospace images with less than 1% of inliers.

Matching aerial images of New York of 1925 and 2014. Manmade changes have even affected the appearance of the coastline, but the configuration of the bridges remains the same.

Inliers according to CSVA describe the bridges (SURF + SIFT + CSVA)

Image registration upon the inliers

Another results

Matching under strong appearence changes

Application of CSVA to dynamic scene matching; overall result of matching (left) and third largest cluster separately (right)

Real-time tracking of multiple objects

Registration under season changes (left), registration map against photo (right)

Source code

The c++ implementation of CSVA is opesource. Here is a link to a github repository

Citing

If you find this code usefull please cite one or both of these papers

1. Malashin R.O. Core algorithm for structural verification of keypoint matches. Intelligent Systems Reference Library. Computer Vision in Control Systems-3. 2018. P. 251-286
2. Malashin R. Matching of Aerospace Photographs with the use of Local Features // Journal of Physics: Conference Series - 2014, Vol. 536, No. 1, pp. 012018.

Bibtex

@unknown{unknown, author = {Malashin, Roman}, year = {2018}, month = {01}, pages = {251-286}, title = {Core Algorithm for Structural Verification of Keypoint Matches}, isbn = {978-3-319-67515-2}, booktitle = {Intelligent Systems Reference Library} }

More details

Detailed explanation is given in this paper.

Here is some explanation without diving into details.

CSVA

“Core” structural verification algorithm (CSVA) is proposed for usage in the sever conditions of viewpoint, lighting changes for a variety of applications. The main idea is to cluster features whose parameters agree in similarity transform space. You can think about the algorithm as (and in fact it is) as the improved and generalized version of Hough Clustering described in SIFT paper by D.Lowe.

Outlier elimination in keypoints-based methods

Outlier elimination is a crucial stage in keypoints-based methods (SIFT, SURF, BRISK) especially in extreme conditions, for example when matching indoor scenes under severe viewpoint and lightning changes or when matching of aerial and cosmic photographs with strong appearance changes caused by season, day-time and viewpoint variation.

CSVA referes to the IV stage in the next figure

Stages of keypoint-based methods

Algorithm pipeline

The proposed algorithm pipeline involves:

1. Many-to-one matches exclusion.
2. The improved Hough Clustering of keypoint matches.
3. Cluster verification procedure based on modified RANSAC.
4. Estimation of the probability that obtained solution is correc.

This site is open source. Improve this page.