Vehicle count and classification with CUDA, OpenCV and particle filters

Not so long ago we (Vicomtech) did a significant effort to complete one of our best works in computer vision. The result is a system for traffic surveillance which uses a single camera, a single PC (CPU + GPU) that is able to count and classify vehicles at high speed.

You can find a summary video here:

As a result we published a number of papers, one of them in the prestigious IEEE Transactions on Intelligent Transportation Systems, and another one in the EURASIP Journal on Advances in Signal Processing.

Some of the key points of the work were the use of a multicue background substraction method that we implemented in real-time thanks to the use of CUDA, and a particle filter for the accurate estimation of the volume of vehicles.

Hope you like it!

This entry was posted in Computer vision, OpenCV and tagged , , , , , , , , . Bookmark the permalink.

26 Responses to Vehicle count and classification with CUDA, OpenCV and particle filters

  1. Subhasis says:

    Hi Marcos,
    I am working on Human tracking based on Particle filter for my masters thesis. Just wanted to know if your experiment with tracking of cars will also fit into my problem scenario ?..And I want to track a region of an image which is not exactly an RGB image. If you believe so, can I have the source code? I would be glad to include your publication reference in my work.

    • Hi,
      Actually I have nothing yet to publish about human tracking, although I’ve seen many times that the SIR particle filter has been used for object detection (including humans) using histogram distances as likelihood.
      Take a look to the work by Rob Hess (http://blogs.oregonstate.edu/hess/code/particles/), I think he shares some code that illustrates well this type of approach.
      Best regards,
      Marcos

      • Subhasis says:

        Thanks a lot Marcos for responding. I actually had a look at the website from Rob Hess. But The download link for the source code seems not valid. As it pops up wit an error hey publish message “The requested URL /~hess/downloads/track.tar.gz was not found on this server.” . I wrote a mail. But have not got any response. If you have some pointers where they publish some code of object tracking using particle filter, it would be really helpful.
        Thanks and Regards,
        SUbhasis

  2. aparna says:

    Hello,
    Where did you get the 3d models for the vehicles. Is there a known repository I can use. Thanks

    • Hi!
      Well, I just created them by myself. The models where cuboids with standards widthxlengthxheight measurements. I used a total number of 20 if I am not wrong, going from small cars to large trucks and trailers.
      Kind regards!
      Marcos

  3. Yeng K. Prosper says:

    Hello Marcos,
    Am really impressed with your work. Congrats! I should be extremely grateful if you can please help me with your source code or any source code. I am working on my MIT project work which i am to blend Hough algorithm with vehicle classification for toll fee collection. I want to base the vehicle classification on width,height, length just like your own.

    Thanks in advance

    • Hi!
      Thanks for writing!
      I can not provide the source code of this solución because I don’t hace the intelectual property which is in the hands of a customer.
      However, I am certain you can find interesting the paper I published in EURASIP in 2012. You can find the PDF for free in the corresponding link in the Publication page.
      Kind regards!!

  4. plka says:

    Hi Macros,
    I am working on vehicle counting as well as classifying the emergency vehicle like ambulance. Can you please help me out for that? I am stucked withe code for detection in which two vehicles moving asides is detected as a one whole object.
    If you can help me than it will be very usefull for me. I am a student and this is a part of my dissertation.
    Thank you

    • Hi!
      You need an additional layer upon the detector, in which you inject rules to split, merge, create or delete detections. Also, if you have information on the perspective, you can estimate the expected size of observations at each image position.
      Good luck with your dissertation!
      Regards,

      Marcos

  5. Rohollah yousefpour says:

    Dear Marcos

    I am Rohollah Yousefpour an Assistance Professor in University of Mazandaran. I study your paper about vehicle classification and enjoy it very much. It is very inserting paper.
    In my my case the speed of vehicle in the gate is very slow and the background subtraction can not detect vehicle. I have some questions:

    1. How i detect vehicle in entrance of a gate?

    2. When a new vehicle enters the , how its classification is obtained from boxes?

    Best regards,
    Rohollah

    • Dear Rohollah,

      Apologies for answering this late.
      Straight to the point:
      -If vehicles are slow you will probably need a segmentation based on optical flow (you can form blobs of similar flow pixels). You can google for MBH (Motion boundary histograms).
      -To classify from blobs or boxes, you need to calibrate the scene. This way you can convert from 2D to 3D measurements (with some uncertainty).
      Regards,
      Marcos

      • Rohollah says:

        Dear Marcos

        Tank you very much for your attention.

        I use the dense optical flow. The small and medium vehicle is detected very good.
        but for large vehicle for example trailer with 15 m length, some parts of vehicle is not recognized. How we can solve this problem?

        best regards
        Rohollah

      • Hi!
        This is an unsolved problem, specially, if large vehicles do not fit entirely into the image.
        We applied a number of expert rules to determine the presence of such vehicles, and to be able to not confuse them with several other smaller vehicles.
        Regards,
        Marcos

  6. Aparna says:

    Hello,
    Do you have results for type of vehicles rather than heavy and light vehicles mentioned I the paper. Thanks

  7. Grazioso says:

    Dear Marcos,
    I work as planner and consultant for public administration in the transportation and mobility fields.
    Your work is very interesting. For the activities related to the vehicle counting I use radar instruments, but in many cases video application could be very useful, especially for the identification of vehicle maneuvers.
    I have some question. Did you make some test with long time of video acquisition? When the light condition change during the day. Did you make test during the night?
    How could I test the Vicomtech application for count, classify and track the vehicles?
    I would like comparing data acquired by radar hardware with data acquired by video source and post processed using your application.
    Thank you for the informations that you can give me.

    Grazioso

    • Hi:
      Apologies for answering this late.
      Yes, we have provided computer vision solutions that work 24/7 detecting, counting and classifying vehicles, including all possible weather and illumination conditions.
      For potential commercial projects with Vicomtech, please send an e-mail to its@vicomtech.org, so we can start a dicussion thread to see what are your requirements and if we can meet them.
      Regards,
      Marcos

    • Aparna says:

      Hello Grazioso, You mentioned vehicles vehicle maneuvers, I would like to understand what kind of vehicle maneuvers are of interest to you.

      • Grazioso says:

        Hello Aparna. When I refer to counting I mean the measure of vehicles that transit through a single section. When I refer to counting of maneuvers I mean the count of vehicles number that transits between a couple of sections (from/to). To collect the maneuver data I need of both vehicles tracking that vehicles counting.
        Using my radar instruments I can’t track, but counting only.

  8. Aparna says:

    Thanks for explaining….that is interesting

  9. tenosca says:

    Hey marcosnietodoncel I should confess your work is jaw dropping, I am developing something similar to yours but its far from yours, firstly how do you manage to segment like that? how do you manage not to include shadows in your bounding rectangles? no matter how much I try solving the background removal problem I never manage to remove shadows and shadows are number 1 at causing problems, if its a sunny day the systems starts merging blobs, I use basic MOG2, then Kalman Filter to reduce chances of false tracking, please help me. thanks.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s