JavaCV IntroExamples

February 2014
ECS522 / ECS744 - Graphical User

Interfaces - 2013/14 1
From OpenCV to JavaCV
An introduction
Topics
Introduction & overview
Basic JavaCV structures & examples
Image I/O
Basic image operations
Examples
Example code & how to use them
February 2014
OpenCV
OpenCV is an Image Processing library created
by Intel and maintained by Willow Garage
Available for C, C++, and Python
Open Source and free
http://opencv.org/
Overview
OpenCV (Open Source Computer Vision)
a library of several hundred algorithms for computer
vision and video analysis
started in the late 90s as a C library
in version 2 a C++ API was added
JavaCV is a wrapper
it allows accessing the OpenCV library directly from
within Java Virtual Machine (JVM) and Android
platform
February 2014
Overview
See the additional lecture notes on QMPlus for
OpenCV & JavaCV
They cover
Installation
Configuration
Access @ ITL
Online documentation
Help
JavaCV project files & examples
http://code.google.com/p/javacv/downloads/detail?name=javacv-0.7-src.zip
February 2014
JavaCV: Basic Structures
Size - 2D size structure
IplImage - image object
Point, Point2f - 2D Point
Rect - 2D rectangle object
Basic Structures: cvSize
2D size structure
- int width, height;
OpenCV version of the structure
Example: CvSize img_size = cvGetSize(image);
img_size.width(), img_size.height()
February 2014
Basic Structures: IplImage
The primary data structure in JavaCV is the IplImage
object
It stores images and their components
Main items
rows, cols - length and width(int)
channels - 1: grayscale, 3: BGR
depth: CV_<depth>C<num chan>
For more information see the documentation
Representing images
A grey-level image can be represented as a
matrix
each cell representing a pixel of the image
cell values usually in the range of [0-255]
pixel values are stored row by row
February 2014
Representing images
A colour image is represented with its
components
[0-255] [0-255] [0-255]
black: ( 0, 0 , 0 )
white: ( 255, 255 , 255 )
Basic Structures: IplImage
http://opencv.willowgarage.com/documentation/basic_structures.html
February 2014
IplImage: example
cvCreateImage creates interleaved images
the channels are interleaved
usual data layout of a colour image is b0 g0 r0 b1 g1 r1
...
Example
IplImage Image = cvLoadImage(filename);
IplImage GrayImage = cvCreateImage(cvGetSize(Image), IPL_DEPTH_8U, 1);
IplImage ColourImage = cvCreateImage(cvGetSize(Image), IPL_DEPTH_8U, 3);
...
IplImage.width()
IplImage.height()
Image types
Image type is a very important aspect of
OpenCV/JavaCV
represented as CV_<Datatype>C<# Channels>
type will affect what values a pixel can have
Example Datatypes/ Depths
February 2014
Pixel types
How the image is represented in data
BGR
The default color for capturing from camera (normal 3 channel color)
HSV
Hue is color, Saturation is amount, Value is lightness (3 channels)
GRAYSCALE - Gray values, Single channel
JavaCV requires images to be in BGR or Grayscale for showing or saving
Converting colour spaces
cvtColor( image, image, code)
Codes
CV_<colorspace>2<colorspace>
Examples
CV_BGR2GRAY
CV_BGR2HSV
CV_BGR2LUV
February 2014
Basic Structures: cvPoint
2D point object
int x_coordinate,y_coordinate;
com.googlecode.javacv.jna.cxcore.CvPoint
Example:CvPoint p1; p1.x()=5; p1.y()=20;
Basic Structures: cvPoint
2D point
February 2014
Basic Structures: cvRect
2D rectangle structure
int x, y, width, height;
x co-ordinate of the top left corner
y co-ordinate of the top left corner
width and height
Example:CvRect box;
box.x()
box.y()
box.width()
box.height()
CvRect rect2=cvRect(box.x,box.y,box.width,box.height);
cvSetImageROI(image, rect2);
Basic Structures: cvRect
2D rectangle object
February 2014
Basic Structures: cvMat
A multi-channel 2D matrix
Example:
CvMat imageMat1 = cvCreateMat(imageRows, imageCols, CV_8UC1);
Basic Structures: cvMat
A multi-channel 2D matrix
February 2014
Image I/O
Loading an image from a file
Displaying an image
Saving an image to a file
Grabbing a frame from camera
Loading/saving images
JavaCV methods for loading and saving images
are based on OpenCV
com.googlecode.javacv.cpp.opencv_highgui
Images can be loaded as IplImage using
cvLoadImage
cvLoadImage usually takes one parameter
file name (with path information if different from current
directory)
February 2014
If image cannot be loaded cvLoadImage will
return null
check what cvLoadImage returns & whether image
can/cannot be loaded
Example:
IplImage imgA = cvLoadImage("image0.png", CV_LOAD_IMAGE_GRAYSCALE);
IplImage imgB = cvLoadImage("image1.png);
cvSaveImage(filename, image)
saves the image to the specified file
image format is chosen based on the filename extension
the method returns an error code
February 2014
Displaying images
Simplest way to display image using JavaCV is
to use cvShowImage
It shows the image in a window
create an image window named smoothed image
show the image
wait till a key is pressed
Displaying images
Another way to display image using JavaCV is to
use CanvasFrame
February 2014
Accessing Pixel Values
Can be done by representing images as a CvMat object.
CvMat can access pixels in a couple of ways, different ones are
suitable in different situations:
by specifying (x,y) or (column, raw) coordinates, for use with gray
level images
by specifying (x,y,channel) or (column, raw, channel) for color
(multichannel) images
To iterate over pixels and channel values you need to
know
number of columns (CvMat.cols), number of rows (CvMat.rows),
and number of channels (CvMat.channels)
Remember: pixel values are stored row by row
Accessing pixel values: Example
IplImage image1;
CvMat imageMat1 = image1.asCvMat();
int imageCols = imageMat1.cols();
int imageRows = imageMat1.rows();
CvMat imageMat2 = cvCreateMat(imageRows, imageCols, CV_8UC1);
// Type matches IPL_DEPTH_8U
cvSetZero(imageMat2 );
for(int irow = 0; irow < imageRows; irow++)
for(int icol = 0; icol < imageCols; icol++)
imageMat2.put(irow, icol, (int)
imageMat1.get(irow,icol));
IplImage image2= cvCreateImage(cvSize(imageCols, imageRows),
IPL_DEPTH_8U, 1);
image2= imageMat2.asIplImage();
What does this
code do?
February 2014
Capturing image
If you don't change any settings, you should get
images in 24 bits per pixel BGR format
i.e.: 8 bits per channel
Remember that rows is your y coordinate and
that cols is your x coordinate.
Example:
// define a grabber
OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(0);
// start the grabber
grabber.start();
// capture an image
IplImage originalImage = grabber.grab();
Drawing stuff
At times you will need to draw stuff onto the
JavaCV image
this can be done using simple functions
void circle(image, Point(x,y),int rad, CV_BGR(b,g,r), int thickness=1)
void ellipse(image, RotatedRect box, CV_BGR(b,g,r), int thickness=1)
void line(image, Point(x,y), Point(x,y), CV_BGR(b,g,r), int thickness=
1)
void rectangle(img, Point(x,y), Point(x,y), CV_BGR(b,g,r), int
thickness)
NOTE: negative thickness will fill in the rectangle
or using other Java functions
February 2014
Drawing functions: cvLine
void cvLine(img, CvPoint pt1, CvPoint pt2, CvScalar color,
int thickness=1, int lineType=8, int shift=0)
Draws a line segment between pt1 and pt2 points in the image
Parameters:
img The image
pt1 First point of the line segment
pt2 Second point of the line segment
color Line color
thickness Line thickness
lineType
8 - (or omitted) 8-connected line
4 - 4-connected line
CV_AA - antialiased line
shift Number of fractional bits in the point coordinates
Example:
cvLine(image, p0, p1, CvScalar.RED, 2, 8, 0);
http://opencv.willowgarage.com/documentation/basic_structures.html#cvscalar
Drawing functions: cvRectangle
void cvRectangle(img, CvPoint pt1, CvPoint pt2, CvScalar
color, int thickness=1, int lineType=8, int shift=0)
Draws a rectangle with two opposite corners pt1 and pt2
Parameters:
img Image
pt1 One of the rectangles vertices
pt2 Opposite rectangle vertex
color Line color (RGB) or brightness (grayscale image)
thickness Thickness of lines that make up the rectangle. Negative values,
e.g., CV_FILLED, cause the function to draw a filled rectangle.
lineType Type of the line, see Line description
shift Number of fractional bits in the point coordinates
Example:
cvLine(image, p0, p1, CvScalar.RED, 3, CV_AA, 0);
pt1
pt2
February 2014
Image thresholding
The simplest approach to segment an image is
using thresholding
If Img (irow,icol) > Thrshld then Img(irow,icol) = 255
else Img (irow,icol) = 0
Image thresholding: Example
static IplImage hsvThreshold(IplImage orgImg) {
// 8-bit, 3- color =(RGB)
IplImage imgHSV = cvCreateImage(cvGetSize(orgImg), 8, 3);
System.out.println(cvGetSize(orgImg));
cvCvtColor(orgImg, imgHSV, CV_BGR2HSV);
// 8-bit 1- color = monochrome
IplImage imgThreshold = cvCreateImage(cvGetSize(orgImg), 8, 1);
// cvScalar : ( H , S , V, 0)
// cvInRangeS(imgHsv, min_color,max_color, imgResult);
//search for the color in image
cvInRangeS(imgHSV, cvScalar(hueLowerR, 100, 100, 0), cvScalar(hueUpperR, 255,
255, 0), imgThreshold);
cvReleaseImage(imgHSV);
// return the processed image
return imgThreshold;
}
February 2014
Image smoothing/blurring
Creating an approximation to capture important patterns
in the data, while leaving out noise or other details
pixel values are modified so that individual points (presumably
because of noise) are reduced,
and points that are lower than the adjacent points are increased
leading to a smoother appearance
Image smoothing
Smoothing is used to reduce noise, sharpness of
edges and detail in an image
void cvSmooth(src, dst, int smoothtype=CV_BLUR, int param1)
Parameters
src The source image
dst The destination image
smoothtype type of the smoothing:
CV_BLUR_NO_SCALE - linear convolution by box kernel (param1 x param1) with all 1's
CV_BLUR - linear convolution by box kernel (param1 x param1) with all 1's and scaling of 1/(param1 x param1)
CV_GAUSSIAN linear convolution with a Gaussian kernel
CV_MEDIAN median filter with a \texttt{param1}\times\texttt{param1} square aperture
CV_BILATERAL bilateral filter with a \texttt{param1}\times\texttt{param1} square aperture, param1 - parameter of
the smoothing operation, the aperture width.
Example:
cvSmooth(src_image, dst_image, CV_BLUR, 7);
February 2014
Image smoothing
OpenCV version of the function
http://opencv.willowgarage.com/documentation/c/image_filtering.html
Using JavaCV for your CW
Capture image
Process image
Higher level
analysis
Use result for
decision making
& interaction
Start/ Create
Save
Open
Move
Copy/Paste
Play
... etc.
E.g.: Location
& center of red
object
detected
Provided via
Code
Examples
Help in labs
camera
February 2014
Using JavaCV for your CW
Capture image
Process image
Higher level
analysis
Use result for
decision making
& interaction
Start/ Create
Save
Open
Move
Copy/Paste
Play
... etc.
E.g.: Location
& center of red
object
detected
Provided via
Code
Examples
Help in labs
WHAT
YOU
WILL BE
DOING!
camera
Lets get started!
LoadImg.java
1.Import the JavaCV related files
2.Try to load an image file
3.Smooth/blur the image
4.Save the image back to disk
February 2014
Example 1: LoadImg.java
LoadImg.java
1.Import the JavaCV related files
2.Try to load an image file example.jpg
3.Smooth the image
You have already tested/worked on this at ITL!
Example 2: CaptureImg.java
Goal: Grab an image
from camera
1.Import the relevant files
2.Define a grabber
3.Grab a frame (image)
continuously
4.Save grabbed image to
disk
5.Show grabbed image
You have already tested/worked on this at ITL!
February 2014
Example 2: CaptureImg.java
What happens if you dont use
if (img != null)
cvWaitKey(5)
What happens if you try to close the window?
Hint: Make sure that your code checks if image is successfully
grabbed & reacts to closing the window.
Perhaps use a KeyListener?
Example 3: ApplyImageOperations.java
Goal: apply a number of operations on a loaded image
Takes an image as input
Clones it into a new image
Returns the new image
February 2014
1. Prints the properties
of the loaded image
2. Converts it to gray-
scale
3. Clones the gray-scale
image into a new
image
4. Skews the gray image
by a given angle
Hint: Play with SkewGrayImage
parameter(s) to see what effect it
has on the result.
February 2014
Example 4: ImagePanel.java
Goal: integrate IplImage & Java image
1.Define & load an image JavaCV style
2.Convert iplImage to Java's BufferedImage Class
3.Start new JPanel with the BufferedImage
4.Draw an ellipse on the image and display it
original image loaded from file
painted image displayed on a panel
frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
How can you use this?
ImagePanel.java
When you need to have objects overlaid on top
of the image (e.g. captured from the camera)
to visualise the output of your programme
to display both the player and the objects (that are
appearing, falling, moving, changing, etc.)
February 2014
Example 5: Ex2MyFirstGUIAppJava
A simple GUI app using Java & JavaCV
JavaCV can add to IplImage a method
getBufferedImage
to convert OpenCV data to Java's
java.awt.imageBufferedImage that can be displayed using
standard Java approach
See the example in Ex2MyFirstGUIAppJava.java
Code by Jarek Sacha
Ex2MyFirstGUIAppJava
Opening or capturing images while also creating
the user interface
Adding panel, buttons, file chooser etc.
Open image, flip it upside down, swap red and blue
channels etc.
February 2014
Example 6: ObjectPositionDetect.java
Goal: detect coloured (e.g. RED) regions/objects
by using HSV colour space
defining lower and upper thresholds
thresholding the image
Example 6: ObjectPositionDetect.java
Goal: detect coloured (e.g. RED) regions/objects
1. Start the grabber
2. Threshold the original image using hsvThreshold
this returns a binary image (black & white)
3. Use cvFindContours to find the contours in the
binary image
i.e., contours of the white areas
4. Draw a GREEN rectangle around each contour
found
Hint: Play with hueLowerR & hueUpperR parameters to see
what effect they have on the result.
February 2014
ObjectPositionDetect.java
Using a red coloured pen/object/glove
E.g.
Navigation using a red-coloured object
static gesture imitation using a red-coloured glove
to play a game or enable painting based on the
movement of the red object

Example 7: detectNplay.java
Goal: To obtain dynamic sound output using dynamic visual
input
As long as frame is being captured from camera
1. Detect the red objects
2. Take the starting point (column value) of the 1
st
detected object (its
contour)
3. Pick a sound to be played: PianoA.aif
4. Change the pitch of the sound using the starting point (column
value) of the detected red object
PitchRatioCalculator.semitoneRatio(440, br.x());
5. Do the above for the next detected red object
February 2014
Example 7: detectNplay.java
As long as frame is being captured from camera
1. Detect the red objects
2. Take the starting point (column value) of the 1
st
detected object (its
contour)
3. Pick a sound to be played: PianoA.aif
4. Change the pitch of the sound using the starting point (column
value) of the detected red object
PitchRatioCalculator.semitoneRatio(440, br.x());
5. Do the above for the next detected red object
Hint: Pick another sound, and use the row value, to see
how the output would change.
Alternative / Advance stuff
February 2014
Example 8: MotionDetector.java
2. Smooth the grabbed image
3. Convert the image to gray scale
4. If there is a previous image grabbed
Take the difference between the current image and the previous
image to detect change
Threshold the resulting difference
Hint: Play with cvSmooth parameters and
cvThreshold parameters, to see what effect
they have on the result.
MotionDetector.java
To detect where there is motion in the room (or
of body, arms, hands) and how much
for obtaining input purely based on motion
E.g., for games played by motion of hands
http://www.youtube.com/watch?v=Li_iCYl2ZQs http://www.youtube.com/watch?v=ddNvNJXwYxU
February 2014
Example 8: MotionDetector.java
Original code can be found -- javacv\samples
Example 9: OpticalFlow.java
Goal: detect key or good features (e.g. corners,
edges of hands) and find where they move
2. Take two consecutive frames & convert them to gray
scale
3. Use cvGoodFeaturesToTrack to find the good
features in the gray scale image
4. Use cvCalcOpticalFlowPyrLK to find the
displacement of key features between the
consecutive frames
5. Draw lines to show where the features moved
February 2014
Example 9: OpticalFlow.java
Hint: Play with
MAX_CORNERS
win_size
parameters to see what effect they have on the result.
OpticalFlow.java
To detect dominant motion (e.g. hand
up/down) in a specific region in a specific
direction
Use it for e.g., games played by motion of the hands
http://www.youtube.com/watch?v=U4taMDEozCs
February 2014
Hints
@ ITL check if cameras are connected
The examples are provided for multiple purposes,
pick the parts that you need for your coursework
If the provided example codes do not work as
desired
check/modify lighting conditions
too dark/too light?
avoid cluttered background
use simple dark background
If you have doubts as to which code to use for
what, ask us!
Remember!
Vision technology is not perfect
Play with the available parameters, e.g.:
Smoothing window size
Lower and upper bounds for colour detection
Vision technology is not perfect, but is fun
Design your interface to accommodate for the
imperfections as much as possible

JavaCV IntroExamples

Uploaded by

JavaCV IntroExamples

Uploaded by

February 2014

ECS522 / ECS744 - Graphical User

You might also like