OpenCV Tutorial

Using OpenCV with Microsoft Visual Studio .net 2005

Lecturer: Amir hossein khalili

Sharif university of technology
March 2007
 OpenCV

The key features

What is OpenCV? (from the documentation)
(from the documentation)
Cross-Platform API of C functions
OpenCV means Intel® Open
FREE for commercial and non-
Source Computer Vision
commercial uses
Library. It is a collection of C
functions and a few C++
classes that implement some
popular Image Processing and What this means
Computer Vision algorithms. You can take advantage of high
speed implementations of
functions commonly used in
Computer Vision/Image
Processing.
 Overview of OpenCV

CV
Image processing HighGUI
and vision algorithms GUI, Image and Video I/O

Cvaux
CXCORE Auxiliary
basic structures and algoritms, (experimental)
XML support, drawing functions OpenCv
functions
CvCam
cross-platform
module for
processing video
ML
stream from digital
Machine
Learning video cameras
methods
 OpenCV

How to obtain the library

Available on Sourceforge
http://sourceforge.net/projects/opencvlibrary/
(Or use your favorite search engine)

How to install the library

(On Windows)

Download and Install the

Executable
 Configuring MSVS .net 2005
Creating the Project
A project is initially created by selecting:
File -> New -> Project

Create a “Win32 Console

Application”

Make it an “Empty Project” by

selecting the box under
“Application Settings”
Configuring MSVS .net 2005
Create the First File
Right Click the “Source Files”
Folder under the project name
(“Test1” in this case)
Add -> Add new Item

Select “C++ file(.cpp)” and

give it a name

Creating a file makes it

possible to set “Additional
Include Directives” in the
C/C++ pane under the project
properties.
Configuring MSVS .net 2005
In order to build projects using
OpenCV the required libraries
and directives must be
included in the project’s
properties

Open the Properties Pane

Right Click the name of the
project and select
“Properties”
(“Test1” in this case)
 Configuring MSVS .net 2005
Set Additional Include
Directives
Under the C/C++ tab select
“General”

Select the “Additional Include

Directories”

Add the full path to each of the

folders which contain “.h” files
required to use OpenCV
Be sure to include trailing “\”

Utilized Directives
D:\OpenCV\cvaux\include\
D:\OpenCV\cxcore\include\
D:\OpenCV\cv\include\
D:\OpenCV\otherlibs\highgui\
D:\OpenCV\otherlibs\cvcam\include\
 Configuring MSVS .net 2005

Utilized Directives

..\..\cvaux\include\
..\..\ cxcore\include\
..\..\cv\include\
..\..\otherlibs\highgui\
..\..\otherlibs\cvcam\include\
 Configuring MSVS .net 2005
Set Additional Dependencies
Under the Linker tab select
“Input”

Select the “Additional

Dependencies”

Add the full path to each of the

“.lib” files required to use
OpenCV
Be sure to keep the paths in
quotes
Utilized Dependencies
“D:\OpenCV\lib\cv.lib“
“D:\OpenCV\lib\cvaux.lib“
“D:\OpenCV\lib\cxcore.lib“
“D:\OpenCV\lib\cvcam.lib“
“D:\OpenCV\lib\highgui.lib"
Configuring MSVS .net 2005

Utilized Dependencies

“..\..\lib\cv.lib“
“..\..\lib\cvaux.lib“
“..\..\lib\cxcore.lib“
“..\..\lib\cvcam.lib“
“..\..\lib\highgui.lib"
 Testing MSVS .net 2005
#include <cv.h>
#include <highgui.h>

Now that the environment is /*

This will pop up a small box with "Welcome to OpenCV"
configured it would be a good as the text.
@author: Amir hossein khalili [email protected]
idea to test it to make sure that imitated from Gavin Page, [email protected]
@date: 1 March 2007
a program will correctly build */
int main( int argc, char** argv ) {
and run. //declare for the height and width of the image
int height = 620;
int width = 440;
//specify the point to place the text
CvPoint pt = cvPoint( height/4, width/2 );
//Create an 8 bit, 3 plane image
IplImage* hw = cvCreateImage(cvSize(height, width), 8,
3);
//Clearing the Image
cvSet(hw,cvScalar(0,0,0));
//initialize the font
CvFont font;
Testing the First Program cvInitFont( &font, CV_FONT_HERSHEY_COMPLEX,
1.0, 1.0, 0, 1, CV_AA);
//place the text on the image using the font
The enclosed code can be cut cvPutText(hw, "Welcome To OpenCV", pt, &font,
CV_RGB(150, 0, 150) );
and pasted into the file created //create the window container
cvNamedWindow("Hello World", 0);
in the project space to test //display the image in the container
cvShowImage("Hello World", hw);
OpenCV //hold the output windows
cvWaitKey(0);
return 0;

}
 Testing MSVS .net 2005
Now that the environment is
configured it would be a good
idea to test it to make sure that
a program will correctly build
and run.

Testing the First Program

The enclosed code can be cut
and pasted into the file created
in the project space to test
OpenCV
 At this point you should have a working
OpenCV project. If the program is not
working you should go back and
carefully recheck the steps.
From here you can explore the
documentation to review the functions
available.

There are also a number of tutorials on the web including:

http://www.site.uottawa.ca/~laganier/tutorial/opencv+directshow/
or you can just search for them

You should also join the OpenCV Community located

at:
http://groups.yahoo.com/group/OpenCV/
As of today there are >15000 members available to
answer questions. There is also a searchable
message board where you can look up previous
queries.
 Memory management
• Why is Managing OpenCV objects Important?
– Video, 30 frames per second
– Each frame is an image
– Images are arrays of pixels
– A 640x480 image is 307,200 pixels
– These must be represented in memory
– How much memory does your machine have?

The metatype CvArr* is used only as a

function parameter to specify that the
function accepts arrays of more than a
void cvResize( const CvArr* src, CvArr* dst, int interpolation ) single type, for example IplImage*,
CvMat* or even CvSeq*. The particular
array type is determined at runtime by
analyzing the first 4 bytes of the header.
 Image data structure
int nChannels
int depth IPL_DEPTH_<bit_depth>(S|U|F)

int width cvCreateImage( CvSize size, int depth, int channels )

int height cvLoadImage( const char* filename, int iscolor=1 )
char* imageData cvReleaseImage( IplImage** image )
int dataOrder
IplImage int origin When allocating IplImage in a loop be
sure to deallocate in the loop as well
int widthStep
int imageSize
struct _IplROI *roi cvSetImageROI( IplImage* image, CvRect rect );

char *imageDataOrigin
int align
cvRect( int x, int y, int width, int height )
char colorModel[4]
 cvLoadImage
Supportted formats:

– Windows bitmaps - BMP, DIB;

– JPEG files - JPEG, JPG, JPE;
– Portable Network Graphics - PNG;
– Portable image format - PBM, PGM, PPM;
– Sun rasters - SR, RAS
– TIFF files - TIFF, TIF.
 Functions
1st & 2nd Image Derivatives. Lines: Canny, Hough. Corners:
Features
Finding, tracking.

In region of interest: Count, Mean, STD, Min, Max, Norm,

Image Statistics
Moments, Hu Moments.

Image Pyramids Power of 2. Color/texture segmentation.

Morphology Erode, dilate, open, close. Gradient, top-hat, black-hat.
Distance Transform Distance Transform

Thresholding Binary, inverse binary, truncated, to zero, to zero inverse.

Flood Fill 4 and 8 connected
Histogram
Manipulation, comparison, backprojection
(recognition
Calc Cov Matrix, Calc Eigen objects, decomp. coeffs.
Eigen Objects
Decomposition and projection.
 Sample Program
• Extracting edges with sobel
void cvSobel( const CvArr* src, CvArr* dst, int xorder, int yorder, int aperture_size=3 );
#include "cv.h"
#include "highgui.h"
int main( int argc, char** argv )
{
char* fileAddress="pic.png";
IplImage* orginalImage = cvLoadImage(fileAddress,0);
cvNamedWindow("Orginal Image");
cvShowImage("Orginal Image", orginalImage);
IplImage* edgeImage =
cvCreateImage(cvGetSize(orginalImage),IPL_DEPTH_16S,1);
cvSobel(orginalImage,edgeImage,0,1);

cvNamedWindow("Edge Image");
cvShowImage("Edge Image", absEdgeImage);
cvWaitKey(0);
cvReleaseImage(&orginalImage);
cvReleaseImage(&edgeImage);
cvDestroyWindow("orginal Image");
cvDestroyWindow("Edge Image");
}
 Accessing image elements
• Assume that you need to access the K-th channel of the pixel at the i-row and
j-th column. The row index is in the range [0-height-1] . The column index is
in the range [0-width-1] . The channel index is in the range [0-nchannel-1] .

Indirect access Onother direct access

CvScalar s; int height = img->height;
s=cvGet2D(img,i,j); int width = img->width;
Int value = s.val[k]; int step = img->widthStep/sizeof(float);
s.val[k]=111; int channels = img->nChannels;
cvSet2D(img,i,j,s); TYPE * data = (TYPE *)img->imageData;
data[i*step+j*channels+k] = 111;

Direct access
Value =((TYPE *)(img->imageData + i*img->widthStep))[j*img->nChannels + 0]=111
 Some other useful data structures

CvMatND
CvMat
OpenCV uses the CvMat* as its Multi Dimentional version of CvMat
general purpose matrix structure. It is
managed in an equivalent style
toIplImage* CvSparseMat
SPARSE N-dimensional array
cvCreateMat( int rows, int cols, int type );
cvReleaseMat( CvMat** mat );

CvScalar
4D vector :double val[4] void cvSet( CvArr* arr, CvScalar value, const CvArr* mask=NULL )

CvScalar s = cvScalar(double val0, double val1,

double val2, double val3)
Some other useful data structures
CvMemStorage
CvSeq
Growable 1-D array, Link list, Queue, Growing memory storage
Stack
cvCreateMemStorage( int block_size=0 );
cvCreateSeq( int seq_flags, int header_size, cvClearMemStorage( CvMemStorage* storage )
int elem_size, CvMemStorage* storage );

Points Rectangular dimensions Rectangular dimensions

CvPoint p = cvPoint(int x, int y); CvSize r = cvSize(int width, int height); with offset
CvPoint2D32f p = CvSize2D32f r =
cvPoint2D32f(float x, float y); cvSize2D32f(float width, float height); CvRect r = cvRect(int x, int y, int width,
CvPoint3D32f p = int height);
cvPoint3D32f(float x, float y, float z);
Working with video sequences
From a camera
Initializing
Capture CvCapture* capture = cvCaptureFromCAM(0);

From a file

CvCapture* capture = cvCaptureFromAVI("infile.avi");

Grabbing,
decompressing and cvQueryFrame
returning frame
IplImage* cvQueryFrame( CvCapture* capture );

The returned image

Processing the should not be released
NULL
image or modified by user

cvReleaseCapture
the image captured by the device is
Releasing the allocated /released by the capture function.
capture source There is no need to release it explicitly
cvReleaseCapture(&capture);
 Motion Analysis and Object
Tracking

• Background subtraction
• Motion templates
• Optical flow
• Active contours
• Estimators
 Background subtraction
• describes basic functions that enable building statistical
model of background for its further subtraction.
• Background statistics functions:
9 Average
9 Standard deviation
9 Running average μijt = α ⋅ I ijt + (1 − α ) ⋅ μijt −1 , 0 ≤ α ≤ 1
 Motion templates
• To generate motion template images that can be
used to rapidly determine where a motion occurred,
how it occurred, and in which direction it occurred.
• Object silhouette
• Motion history images
• Motion history gradients
• Motion segmentation algorithm
MHG
silhouette MHI
 Optical Flow
• Block matching technique
• Horn & Schunck technique
• Lucas & Kanade technique
• Pyramidal LK algorithm
• 6DOF (6 degree of freedom) algorithm
 Active Contours
• Snake energy: E = Eint + Eext
• Internal energy: Eint = Econt + Ecurv
• External energy: Eext = Eimg + Econ
• Two external energy types:

Eimg = − I ,
Eimg = − grad ( I ) ,
E = α ⋅ Econt + β ⋅ Ecurv + γ ⋅ Eimg ⇒ min
 Estimators
• Kalman filter
• ConDensation filter
 Saving a video file
CvVideoWriter *writer = 0;
int isColor = 1;
Initializing a int fps = 25; // or 30
video writer int frameW = 640; // 744 for firewire cameras
int frameH = 480; // 480 for firewire cameras
writer=cvCreateVideoWriter("out.avi",
CV_FOURCC('P','I','M','1'),
fps,cvSize(frameW,frameH),isColor);
Writing frames
to video file
IplImage* img = 0;
int nFrames = 50;
for(i=0;i<nFrames;i++)
{
Is there
Img=cvQueryFrame(capture);
more cvWriteFrame(writer,img);
}

Releasing the
video writer cvReleaseVideoWriter(&writer);
 Possible Codecs for saving
Codec fourcc
MPEG-1 CV_FOURCC('P','I','M','1')

motion-jpeg CV_FOURCC('M','J','P','G')

MPEG-4.2 CV_FOURCC('M', 'P', '4', '2')

MPEG-4.3 CV_FOURCC('D', 'I', 'V', '3')

MPEG-4 CV_FOURCC('D', 'I', 'V', 'X')

H263 CV_FOURCC('I', '2', '6', '3')

FLV1 CV_FOURCC('F', 'L', 'V', '1')

A codec code of -1 will open a codec selection window (in windows).

Thank You! Questions?