|
|
|
|
Correlation-based matching of color images with occlusions |
|
|
One of the goals of stereovision is to find the third dimension from two images taken from two different angles. While searching for the third dimension, two other problems occur: calibration and matching. Matching is an important task in computer vision, the accuracy of the three-dimensional reconstruction depending on the accuracy of the matching. The problems of matching are: intensity distortions, noises, untextured areas, foreshortening and occlusions. Our works deals with dense matching of pixels using correlation measures. It also takes into account color images and occlusions. |
Advisor : Alain Crouzil |
Evaluation and comparison of correlation measures - New robust measures
Bibliography about correlation measures
We consider that a correlation measure evaluates the similarity between two data sets: two pixels and their neighbourhoods. Firstly, this work classifies correlation measures into five families. The description of the properties of these measures can help in the choice of a correlation measure. Then, eighteen new robust measures based on robust statistics are proposed : the median absolute deviation, the least median of powers, the least trimmed powers, the smooth median powers deviation, six R-estimators, eight M-estimators. Finally, we set up an evaluation protocol that compares all measures. The results show the most efficient measures: the robust measures.
Publication
Sylvie Chambon, Alain Crouzil. " Similarity measures for image matching despite occlusions in stereo vision ". Pattern Recognition , PR. To appear.
Fusion of classic and robust correlation-based matching
In order to take into account occlusions, we proposed eighteen robust measures based on robust statistics. We showed that classic measure are the most robust near occlusions whereas robust measures are more efficient than classic measures in non-occluded areas. Consequently, we propose new algorithms that use both classic and robust measure to obtain good results on the whole image. We proposed two types of algorithms :
Algorithm 1 - The steps are : to build the disparity maps (left-right and right-left) using the classic measure, to locate occluded pixels with the symmetry constraint, to locate the whole occlusion area and for the whole occlusion area, to calculate new disparities using the robust measure. The problem is how to detect the whole occlusion area.
Algorithm 2 - The two steps are to compute independently two disparity maps using the classic and the robust measures and to merge these maps in order to obtain the final disparity map.
This latter algorithm is the most efficient, however, it is the most expensive.
Sylvie Chambon, Alain Crouzil. " Combination of correlation measures for dense stereo matching ". In International Joint Conference on Computer Vision Theory and Applications, VISAPP, Vilamoura, Portugal, march 2011.
Sylvie Chambon, Alain Crouzil. " Towards correlation-based matching algorithms that are robust near occlusions ". In International Conference on Pattern Recognition - ICPR 2004, p. 20-23, Cambridge, UK, August 2004.
Color correlation measures
Although, the use of color images is more and more frequent in computer vision and can improve the accuracy of stereo matching, few papers present correlation measures using color images. Our purpose is also to take into account color in dense matching using correlation and to adapt our previous work. We have to :
Method 1 - To compute the correlation with each color component and then to merge the results;
Method 2 - To process a principal component analysis and then to compute the correlation with the first principal component;
Methode 3 - To compute the correlation directly with colors.
The results highlight that color always improve matching even if the best color space and the best method are not easy to distinguished. In fact, the choice of the color space and the method depends on the measure. Nevertheless, we can conclude that method 1 or 3 are often the best.
Publication
Michael Bleyer, Sylvie Chambon. " Does Color Really Help in Dense Stereo Matching ? ". In 3D Data Processing Visualization and Transmission, 3DPVT, Paris, France, may 2010.
Michael Bleyer, Sylvie Chambon, Uta Poppe, Margrit Gelautz. " Evaluation of different methods for using colour information in global stereo matching approaches ". In The Congress of the International Society for Photogrammetry and Remote Sensing, Beijing, China, July 2008.
Sylvie Chambon, Alain Crouzil. " Colour correlation-based matching ". In International Journal of Robotics and Automation, IASTED, volume 20(2), p. 78-85, 2005.
Sylvie Chambon, Alain Crouzil. " Color stereo matching using correlation measures ". In Complex Systems Intelligence and Modern Technological Applications - CSIMTA 2004, p. 520-525, Cherbourg, France, September 2004 .
Images with ground truth
In order to accurately evaluate matching methods, we need an evaluation and comparison protocol. So we have to determine :
Until now, our evaluation only use images from Scharstein and Szeliski . We have proposed a new semi-automatic method with plane-based segmentation in order to obtain accurate images with ground truth. This new data can be downloaded here.
Publication
