Hi guys! The vanishing point detection topic has occupied a good part of my (research) life. Indeed I spent quite a long time to finish my PhD whose title was “Detection and tracking of vanishing points in dynamic environments“.
In this post I would like to show a simple yet robust solution for the detection of a single vanishing point in road scenes.
The vanishing point in this scenario can be very usefull to retrieve the camera calibration, to perform some planar homography transformation, to determine a ROI inside the image, etc. Although there are several vanishing points defined by the elements of this scenario (the vertical and horizontal directions of the panels), we want to focus on the vanishing point defined by the lane markings. Note that for curvy roads, the vanishing point does not exist, although you can thought of it as the direction of the tangent on the car position on the curve.
For that reason, we first need to extract the lane markings, which can be done in many, many different ways (thresholding the intensity, connected components, edges, etc). In this post I share one of the fastest I’ve used in my works. It is published in my Springer MVAP paper “Road environment modeling using robust perspective analysis and recursive Bayesian segmentation”, and the code in C++/OpenCV I share here (sorry it’s an image because the <code> </code> html commands seem not to work fine in WordPress):
Applying this filter we get images like the following (note that I have set to black the upper half of the image), where tau is the expected width (in pixels) of the lane markings. For a better performance, this value can be adapted to the perspective of the road, although for this special case this is exactly what we do not have!):
After a proper thresholding we can get something like this:
Although we do have a lot of false positive pixels (the vehicle or lateral elements of the scenario), the following robust stages will find the correct vanishing point.
Using OpenCV, I have found that a quite reliable solution is based on (i) the use of the Hough transform, and (ii) the computation of the intersection of the lines we get.
For the first part, OpenCV has two main options, the Standard Hough Transform (SHT), and the Progressive Probabilistic Hough Transform (PPHT). I use the first because it returns lines and not pairs of points or line segments and although it is a little bit slower, it requires the user to set less parameters and it works fine in most cases. The Hough transform can be applied as:
Where __lmGRAY is the image we obtain from laneMarkingsDetector, and __houghMinLength is the minimum length we require (it should be set according to the image dimensions, something like 30 should work for small images 320 x 240).
The result is a set of lines that visually converge on a small region of the image:
In this simple case there are no strong outliers, i.e. lines that clearly do not intersect at the vanishing point, although we do have a non-negligible intersection error. For cases like this or with more outliers, we can use a RANSAC-like method to find the more likely vanishing point.
(UPDATE: The MSAC class is no longer available as it was, instead, you can download the new MSAC class with a full sample capturing images or video and computing as many vanishing point as desired, both finite or infinite. Please refer to the specific post for more details).
For that purpose I use a variation of RANSAC called MSAC which simply weights inliers according to their cost function (instead of just counting 1 for inliers and 0 for outliers as RANSAC does). I have programmed a very simple version of it, in a C++ class, which only needs two steps:
__msac.init(IMG_WIDTH, IMG_HEIGHT, 1);
// Execution (passing as argument the lines obtained with the Hough transform)
__msac.singleVPEstimation(lines, &number_of_inliers, vanishing_point);
Where __msac is an object of class MSAC, number_of_inliers is an output int that contains the number of inliers MSAC has found to compute the vanishing_point (if you want to play with this, you can go to the Code page from my website although it is not optimized nor commented).
The result is normally a good vanishing point (I have tested it in many, many type of road sequences, and it works fine as long as there are some painted lane markings).
Additionally, I usually compute the vanishing point for a set of time instants, and check if the vanishing point is coherent and steady in time. In case not, I restart the procedure until I find something reliable.
That’s all for today!