Currently, I am participating in an open source project, VCD (Video Content Description), leaded by Vicomtech. Its a metadata language (Python API, JSON schema) for complex labels, including ability to label objects, actions, relations, etc. Its devised to support labels for multi-sensor set-ups (e.g. a set of cameras and LIDAR from an instrumented vehicle).

You can find more info at the VCD webpage.

In my Github page you can find some other repositories I’ve created or participated in:

GCL – Generic C++ Library
LSWMS – Line Segment Weighted Mean-Shift
Graph – Graph Solving Algorithm

There are some legacy repositories in sourceforge, although these are at least 8 to 10 years old (from 2010-2012), so use with caution!


40 Responses to Code

  1. Pingback: Lane markings detection and vanishing point detection with OpenCV | Marcos Nieto's Blog

  2. zoo says:

    Thank you for your code. I have done modifications of [lmfit 5.1] for your vanishing point(MSAC) code as I remove printout flag.

  3. Anil says:

    Hi Marco

    Can you share source code for you lane detection and tracking project ?


    • Hi Anil,
      I will do so soon. I am preparing a new example project that does lane tracking.
      Please be patient, my free time is very limited.
      Kind regards!

      • TaeGeun Kim says:

        Hi. would you send me email? I would like to ask you more about the lane departure application using OpenCv

      • Clemens Schartmüller says:


        Is there any progress on this? I am currently doing my bachelor thesis and experiencing some problems, where lane detection is one of them. I would greatly aprecciate such an example project.
        I’ve also written you a PM on your YouTube-Channel. Please contact me via mail, if you have time and motivation to help. Thank you very much in advance.


      • Hi!
        Not really. Alas, my free time is close to zero. Most of the stuff I publish now is related to new papers I write or projects I lead in my work at Vicomtech-IK4.
        Good luck with your project!

      • Clemens Schartmüller says:

        Thank you for your response. If you still somehow find time to share some code or rough description of your used algorithm, I’d be very glad for it. Just throw me a mail if so.

        Have a nice day.

      • Joe says:

        Hello Marco,
        I am very pining for the lane detection and tracking source code…
        I am waiting for four months for you to share the code,
        Thank you ^_^

      • Hi!
        You are right, I said I was releasing some code, but this is getting more difficult for me nowadays: I am working for my employer in similar matters and thus anything that I can share is what I do at home in my free time, which is very limited now.
        In any case, you can try by yourself. You can start computing the vanishing point (e.g. only at the beginning, or continuously), with that, compose a bird’s-eye view of the road (an homography between the image plane and the road plane), and operate on that image. Detect lane markings applying a row-wise bump filter or any other pixel-wise detector you can find. Then, you need to fit a lane model to the filtered image, using techniques such as Least-Squares, RANSAC for removing outliers, and finally, provide some temporal coherence to smooth the results and be robust against absences of detections, by using a Kalman filter or similar.
        These are the basics. The difficult work is to connect every node in a way the entire chain works.
        Hope you find a way.


  4. Cristian S. says:

    Hello mister Marcos Nieto, I apologize for bothering you. I am a Mechanical Enginner, and I am working on a project that uses OpenCv to detect a specific colour of an object and to estimate the distance from the camera to the detected object using only one camera. At this moment I managed to detect the object, but I don`t know how to estimate the distance. Can you please help me with this problem? Thank you and I’m waiting for your answer.
    P.S. We can talk more on skype if you have a little free time.

    • Hi!
      My free time is very limited, I am sorry.
      Regarding your question, you can estimate the distance to an object in a single image if you do have contextual information of the scene, such as the calibration of the camera and the assumption of an existing plane.
      Try google homography, which can give you an idea on hoy to move from pixels to meters in a given scene.
      Kind regards,


  5. Xie.YZ says:

    Hi Marco
    I watched the video on youtube:, I was fascinated by it’s effects, it’s great, can you make this project as a open source project on github,com or, I’d like taking part in, if you don’t like make it publicly, can you share the source code to me via email, thanks very much.

    • Hi!
      Thx for writing! Unfortunately, I am not able to share that code because it belonged to my employer at the moment of making it (this is some years ago now).
      From time to time I am sharing pieces of code, renewed and improved with respect to that version, which I can make it public.
      I have an ongoing project for making public a lane tracking system but really, my free time is extremely short and it can still need some time to see the light.
      That is for academic or hobby queries. However, I work for a private research centre where I work creating professional computer vision applications. If there is a commercial interest we can still talk about ideas or opportunities.
      Kind regards,


  6. Niks J says:

    Can you tell me the code for drowsiness detection and yawning detection

    • Hi,
      I am sorry, this is an ongoing project within Vicomtech-IK4 and not a personal project I can share with you all.
      If you are interested in a potential collaboration or a commercial interest, please write me a private e-mail and we can discuss it.
      Kind regards,

  7. Ravichandra says:

    Hi Marco
    I am doing project on vision based aircraft runway detection.
    I watched the video on youtube and is a good research work. In my project also i detect hough lines but it will reach vanishing point otherwise only two lines up to Horizon.
    That is the problem i want it like a rectangular shape because i detect the corner point of that please share the code.
    Otherwise send the code of how rectangular shape of hough lines is getting.
    Thanks in advance..

    • Hi,
      I am unsure what you mean with the rectangular shape. If you detect Hough lines, and then compute intersection between lines, you got vanishing points. What is the following step?

  8. bMohit says:

    Hello Marco..
    I am trying to implement visual odometry using absolute orientation method. I am using kitti dataset. I have obtained Rotation(3×3) and translation(3×1) matrices for consecutive images. I just want to know, how to convert these matrices into ‘poses’ and plot ‘map like’ data. I am aware that camera centre is -R’.t.

    • Hi,
      I believe you nearly have it. Visual odometry gives you frame-to-frame relative rotation and translation. So, you can concatenate transforms to obtain the rotation and translation of every frame with respect to the first one. This way you can have a trajectory. If the initial frame has a given offset with respect a desired world coordinate frame, you can add this additional transform to the entire trajectory and you can possibly plot it without problems. I am not expert in visualization nor computer graphics, but some alternatives may include using native OpenGL code, ViZ (, Blender, or whatever suits you well.
      Kind regards,

  9. Brilly says:

    Hi Marco, my name is Brilly. I am a mechatronics student in Indonesia and I am interested to learn more about how to detect vanishing point on your website.

    I have downloaded your code from: . But the problem is I could not really understand about what the code does. Maybe can you give me some of the algorithm about how the code works?

    I also have problem on running the code on my linux because I am just new to linux and I want to explore more on doing the programming on linux. Do you have some steps on how to make your program works on linux?

    Any help would be appreciated, thank you.


    • Hi Brilly,
      Apologies for my late answer.
      I can suggest you to take a look to any paper related to vanishiing point estimation, to get some background about the matter. If you are already doing so, you can take a look to my PhD thesis, to check some details about the methods I am sharing in sourceforge:
      The code is ready to be compiled in linux using CMake, please take a look to the readme files inside.


  10. habib says:

    Hi Marcos Nieto,

    I am trying to find the vanishing point in the video. However, the dependencies are no longer available as you mentioned regarding your posted code. Therefore, I am trying to use lmfit for the same purpose provided it as an alternative. I found it complicated to use it. Could you please elaborate how to use it for finding the vanishing point. That would help me out a lot. Thanks ahead before hearing :)


  11. John Li says:

    Hi Macro,

    In your line detect library, file LSWMS.cpp, line 905 – 907.
    if(x1 < x2) sx = 1; else sx = -1;
    if(y1 < y2) sy = 1; else sy = -1,
    err = dx-dy;

    I think the "sy = -1," should be "sy = -1;".
    Can you confirm?


  12. B says:

    Hi! Thanks for making this code available. I am trying to implement a stereo visual odometry project, but it is driving me insane. I don’t suppose you have any open-source solutions to this that i could take a look at? The Vicomtech one is very impressive! Many thanks!

    • Hi!
      Thanks, I am afraid all I have done about odometry is not open source. Vicomtech has indeed single and stereo visual odometry included in the Viulib libraries, which are still of limited access upon reaching agreements about exploitation and commercial applications.
      Odometry is not easy, but you can still work on reading latest developments, and take a look to ROS ( because they have open source SLAM applications that might be of interest to you.


  13. vivek soni says:

    Do you have any tutorial for detecting vehicles (when camera is also moving ) ?
    Please help me .

  14. shahin says:

    hello marcos
    I need help for code of Adaptive Multi-Cue Background Subtraction for
    Robust Vehicle Counting and Classification
    can you help me?

  15. Brylle says:

    Hi Marco. do you have any idea how to compute the distance from the camera to the object detected?Thank you and sorry for my English

    • Hi:
      Yes, but this is not a simple task! There are several ways to do it.
      One is to assume that the object is on a planar surface. In that case, you can compute the homography between the image plane and the world plane, as long as you have a reference to solve scale (for instance, 4 known points in the world plane for which you can find their projection in the image).
      Other approaches imply the calibration of the intrinsic and extrinsic parameters of the camera. Even in that case, any point in the image is projected as a ray that passes through the optical center, so you will need more information about the object.
      This is a well-known topic in the literature of computer vision. I recommend you to take a look to books about projective geometry (google “projective geometry hartley zisserman” and you will find the pdf).

  16. Chanmi You says:

    Hi, Marco. I appreciate you for uploading this code for free.
    I’m implementing forward vehicle distance estimation, which includes vanishing point detection.
    So I need this code and I want to ask permission to you.
    Could I use this code for commercial purpose?
    Chanmi You.

    • Hi,
      Sure, you can use the code as you want. Honestly, the code I am sharing has been conceived as an illustrative example, not application-ready code, so without optimizations nor support.
      Good luck!

  17. Nilantha Saluwadana says:

    Hi, Marco, Can you share source code for you lane detection and tracking project ? or Please give me a tip to do that.

    • Hi,
      Unfortunately, I haven’t published code of entire ADAS applications. Basically, I don’t have enough free time to do this because I would need to prepare the code to be readable, and prepare kind of a tutorial. I work for Vicomtech-IK4, who retain all the IPR.
      Nevertheless, you can probably put an eye to OpenCV and the contrib modules, which look to start having ADAS examples.

  18. Rolba says:

    Hi Marco.
    I am having similar project made by ma self after working hours. And I meet one problem which is not exactly a very big issue but I can see it.

    Let me describe. Wen I make IPM (in my case it’s not exactly the same but the idea is the same :). I noticed that on straight and ideal road, the road lines are parallel – that is good!. But when the road is having some potholes and bumps than some vertical vibrations components occurs and after that the lines after IMP are not parallel, the have some angle between dependent on the force was caused by road defects. Have you tried to deal with it?

    My idea is to have g-sensor on cam and add some corrections in vertical axis for wrap transformation. But I have also an idea to make it by code and add wrap transformation correction coefficient knowind that road has always constant width (what can be calculated using white nearest road lines).

    I hope that you get the point ;).

    • Hi,
      Yes, I see your point. Actually this is something I tried to tackle time ago, in what we called “Stabilization of IPM”. Back in the 2000’s there were some movement around this idea, because, yes, IPM images are distorted by non-zero pitch angles of the car, which occur in bumps, slopes, curves, etc.
      The solution could be to find these lines that are no longer parallel in your IPM, and find a second homography that map them back to parallel. Equivalently, you can try to estimate the pitch angle for every frame, and create an IPM which is adapted for each frame.
      However, this could be bad for two reasons: (i) you need extra computational load to find this additional transformation, (ii) creating a new IPM for each image disables the advantage of using LUT (or remap if you’re using OpenCV).
      My advice is to not even build your IPM if possible: operate on the original image, get the points, lines or other shapes there, and transform them into the IPM and work with them in that domain. Make your algorithm robust against pitch errors. This way your algorithm will be faster, at the cost of moving the intelligence to the analysis step, not to the image processing.
      Hope this can help, good luck!

  19. tolga says:

    Hi marcus great work out there;
    I tired your line segmentation codes on some videos however it performance very poor even on a 360p (and hence 720p) video, which is not suitable for real time applications. Do you suggest any method to apply these on real time applications? I will also remove lines which are not in my interest.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s