Pre-loading video images in std::vector

Have you ever thought to load all images of a video file into a std::vector just to see if your program gets faster than to iteratively read the images from the file?

Well, I did. The hypothesis is that accessing the video info from the file all together at the beginning might at the end be faster that doing it iteratively interlaced with the processing of the images.

Normally I use the following code:

#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
using namespace cv;
void main()
{
  VideoCapture video;
  if( !video.open("myFile.avi") )
    return 1;
  Mat image;
  for(;;)
  {
    video >> image;
    if( image.empty() )
      break;
    // Do some process
    ...
  }
}

While, you can as well do this:

#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
using namespace cv;
void main()
{
  VideoCapture video;
  if( !video.open("myFile.avi") )
    return 1;
  int numFrames = static_cast<int>(video.get(CV_CAP_PROP_FRAME_COUNT));
  vector<Mat> videoFrames;
  videoFrames.reserve( numFrames );
  for( ; ; )
  {
    video >> inputImg;			
    if( inputImg.empty() )
      break;
    Mat aux;
    inputImg.copyTo( aux );
    videoFrames.push_back( aux );
  }
  Mat image;
  for(size_t i=0; i<videoFrames.size(); ++i)
  {
    image = videoFrames[i];
    // Do some process
    ...
  }
}

Of course, the comparison depends on your machine, if it does have enough RAM memory to host the entire video. I believe also in some other variables like your HD type, and so on, but I am not entering in such details.
For a short video of 2792, 376×240 pixels, fourcc = 1145656920, I tried both options with the following result:
-Using VideoCapture: 4451 (ms) = 1.59 (ms/frame)
-Using std::vector: 5568 (ms) (load time = 1666 (ms) + 3902 (ms) processing) = 0.59 (ms/frame) + 1.39 (ms/frame) = 1.99 (ms/frame)

So, there are two reads here. On the one hand, the sum of time make me think that using VideoCapture for one loop processes is still a good option, since the total time is 1.59 (ms/frame), while using vector increases the total time to 1.99 (ms/frame) mainly because of the overhead of loading the images.
On the other hand, if you are about to run several different algorithms on the same video in a separate manner, you gain a little bit because you only have to do the loading once (and it is done very quickly), and accessing the (already decoded) images from memory is better than from file (1.39 ms vs 1.59 ms).

That’s all!
Kind regards

Marcos

Advertisements
Posted in OpenCV | Tagged , , | Leave a comment

Using OpenCV 3.0.0alpha with CMake in Windows 7

Hi all,
I was just curious about how much effort would take to download, install and try the OpenCV 3.0.0alpha version that has been around for a little while (http://opencv.org/opencv-3-0-alpha.html). The motivation is that it looks that we will have soon a beta version, and perhaps a release version somewhere at the end of 2014 or beginning of 2015. I will definitely like to have acostumed myself to whatever changes OpenCV 3.0 implies compared to OpenCV 2.X.

Downloading OpenCV from git repository
So the first thing is downloading this version, and the suggested way to do it is with the git repository. So you need to install the git tool if you do not have it already. You can download it from here: http://msysgit.github.io/
During installation I left everything default and created an user and password. Once done, you have a Git bash application from which you can type the download commands. Actually, the first time, in git, this is called “cloning” the repository. You can do it from here: https://github.com/Itseez/opencv.git

I also downloaded the extras path: git://code.opencv.org/opencv_extra.git
And the contrib folder, which contains the contributions from external contributors (appart from itseez): http://github.com/itseez/opencv_contrib

Building OpenCV with CMake
With CMake you can very easily build OpenCV in your system. You can follow any previous tutorial to do so (like the one I did some time ago). If you are unfamiliar with CMake, please consider it a must if you are interested in programming computer vision applications.

Using OpenCV 3.0.0alpha in your projects

First thing, is that the name of the CMake list variable that holds the paths to the include directories has changed from ${OPENCV_INCLUDE_DIRS} to ${OpenCV_INCLUDE_DIRS}. Yes, well, not a big change, but here I had my first problem until I realised what was happening.

The second problem was that some OpenCV macros has been moved to new files so I had to search for them and #include the files. For instance, you would probably need to include the following:

#include "opencv2/imgproc/imgproc_c.h" // for CV_RGB
#include "opencv2/videoio/videoio_c.h" // for CV_CAP_PROP_FRAME_WIDTH, etc. when capturing a video

I also got the strange problem that the ippicv.lib library was not found, even if there were no warnings nor problems at config time. This is something I will investigate later. Just removed this ippicv.lib from the Linker menu in Visual Studio and everything worked just fine.

Excited about discovering the new features of OpenCV 3.0.0alpha!

Regards,

Marcos

Posted in CMake, Computer vision, OpenCV | Tagged , , | Leave a comment

ITS World Conference – Detroit 6-11 Septembre, 2014

Hi!

EDIT: I’ve just come back, and everything was incredible there. Many big OEMs showed their vision on future vehicles, and demonstrated some autonomous driving capabilities. We had a booth were we demonstrated our capabilities in computer vision applied to ADAS, ITS, etc.

Marcos Nieto at ITSWC2014

Marcos Nieto at Vicomtech’s booth in ITSWC Detroit 2014

I will attend the Intelligent Transport System World Conference in Detroit next 6-11 Septembre (ITSWC2014). We will have a booth there presenting some works about ADAS (Advanced Driver Assistance Systems) and our computer vision technology in general.

There, I will give a short talk about the methodology we follow when creating computer vision algorithms for embedded platforms for automotive applications.

Here you are a preview of the slides I will use there:

And the reference of the paper I am authoring.

  • G. Velez, O. Senderos, M. Nieto, O. Otaegui, and G. Van Cutsem, “Implementation of a computer vision based advanced driver assistance system in Tizen IVI,” in Proc. 21st ITS World Congress, Detroit, U.S., 7-11 September 2014 (accepted).
  • M. Nieto, J. D. Ortega, O. Otaegui, and A. Cortés, “Optimization of Computer Vision Algorithms in Codesign Methodologies,” in Proc. 21st ITS World Congress, Detroit, U.S., 7-11 September 2014 (accepted).

I will update this post when I come back.

Cheers!

Posted in ADAS, Computer vision | Tagged , , , , | Leave a comment

Creating mosaic of videos using ffmpeg command line

Hi all,

I’ve been looking for a simple solution to create a mosaic of videos more time that I would like to admit. One can do it using OpenCV, or special video editing tools, like Adobe Premier, but I wanted something really specific and simple.

Somebody told me “Maybe ffmpeg can do the job”. Of course!, and it is really straightforward. Here you are an example I just tried, creating a mosaic of two videos of the same size, putting them side by side.

So we have left and right videos:

left  right

And we want to have a single video like this (which can be really useful in Power Point presentations or alike when you want to compare the output of two methods or two videos simultaneously):

mosaic

So the code I used is:

C:\ffmpeg\bin\ffmpeg -i left.avi -i right.avi -filter_complex "nullsrc=size=720x288 [base]; [0:v] setpts=PTS-STARTPTS, scale=360x288 [left]; [1:v] setpts=PTS-STARTPTS, scale=360x288 [right]; [base][left] overlay=shortest=1 [tmp1]; [tmp1][right] overlay=shortest=1:x=360:y=0" -c:v libx264 mosaic.avi

You can download the latest binaries of ffmpeg here.

And you can find more details on this topic in the wiki page.

Cheers!

Posted in Computer vision | Tagged , , | Leave a comment

FP7 TAXISAT project – Driverless car using stereo visual odometry

Hi all,

Recently we have concluded an FP7 project, called the TAXISAT project, in which we worked for building a driverless car which uses GNSS, stereo visual odometry and laser obstacle detection for driving in any kind of environment.

Its main use is the “last mile problem”, such as moving persons in a taxi airport area, entertainment parks, from parkings to stadiums and so on.

We are proud of the results achieved here in Vicomtech-IK4, and willing to continue working on this kind of technologies, especially in stereo vision.

There has been many people involved in this project, but I would like to metion my colleague Leonardo de Maeztu, who has leaded the optimization of the algorithms and the integration and testing stages.

Hope you like it!

Posted in ADAS, Computer vision, FP7 | Tagged , , , , | 1 Comment

Source code Inverse Perspective Mapping C++, OpenCV

Hi all,

Today I bring a very simple code that might be of interest for some of you. It is a C++ class to compute Inverse Perspective Mappings (IPM), or sometimes called bird’s-eye views of a planar surface.

The link to the code: https://sourceforge.net/projects/ipmapping/files/

(I am providing here a CMakeLists.txt and some c++ files, so you need to create a solution with CMake, and the a C++ compiler).

sample

It is nothing else than a plane-to-plane homography, but in my experience it is not that easy to compute. I am using here OpenCV remap functions.

In the example I have hard-coded the necessary four-point correspondences required to compute the homography, but you can compute then as you want. You might use vanishing points, camera calibration information, or any other source.

Of course this type of classes can get really useful when you add them information about the camera calibration.

Hope it is useful!

Kind regards,

Posted in ADAS, Computer vision, OpenCV | Tagged , , , , , | 46 Comments

Mastering OpenCV with Practical Computer Vision Projects [eBook]

Hi all!

OpenCV is a must when talking about computer vision, rapid prototyping, and open source solutions. Many people is interested in learning OpenCV, but sometimes it is a little bit tricky to find the correct way to use the primitives that OpenCV offers.

A great help can be found in the OpenCV documentation and tutorials, but it is more difficult to find good larger examples on the many features OpenCV provides.

An excellent way is to take a look to Mastering OpenCV with Practical Computer Vision Projects by Daniel Lélis. I’ve found this book really amazing. It does not only cover the hot topics around OpenCV (Android, iOS, structure from motion, face modeling, etc.) but it also comes with the source code available, which you can use and see all this cool stuff running in your own PC or smartphone. I really recommend to read this book. The authors and reviewers of the book are first class computer vision and OpenCV experts, not mentioning the high quality of the coding itself, which can be used as a guide on how to fully exploit C++ with OpenCV.

opencv

In summary: if you are learning OpenCV, you have tried some of the examples and you want to give the next step building your own large projects (especially in Android or iOS), you need this book!

Hope you enjoy it!

Posted in OpenCV | Tagged , , , , | Leave a comment