Introduction to OpenCV Computer Vision Luigi De Russis 

OpenCV in brief OpenC… what? 07/11/2013 2 Introduction to OpenCV 

What is OpenCV? 07/11/2013 Introduction to OpenCV 3  Open source Computer Vision library  BSD License  http://opencv.org  Originally developed by Intel  Has more than 2500 optimized algorithms  Supports a lot of different languages  C, C++, Python, Java  but is written natively in C++  Cross platform  also available for Android and iOS 

What it is used for? 07/11/2013 Introduction to OpenCV 4  Human-Computer Interaction (HCI)  Object Identification  Object Recognition  Face Recognition  Gesture Recognition  Motion Tracking  Image Processing  Mobile Robotics  … and so on 

Always ask “why?” Motivations - OpenCV 07/11/2013 5 Introduction to OpenCV 

Why OpenCV? 07/11/2013 Introduction to OpenCV 6 vs. 

OpenCV: pros 07/11/2013 Introduction to OpenCV 7  Specificity  OpenCV was made for image processing  Matlab is quite generic  Speed  30+ frames processed per seconds in real time image processing with OpenCV  around 4-5 frames processed per seconds in real time image processing with Matlab  Efficient  Matlab needs more system resources than OpenCV 

OpenCV: cons 07/11/2013 Introduction to OpenCV 8  Easy of use  Matlab won hands down!  Integrated Development Environment (IDE)  you can use Eclipse, Netbeans, Visual Studio, Qt, XCode, … even a simple text editor for OpenCV  Matlab has is own IDE  Memory management 

OpenCV: pros (final) 07/11/2013 Introduction to OpenCV 9  Price (!)  OpenCV Wrappers  SimpleCV, Emgu CV, ruby-opencv…  More similar to industry-level frameworks 

When Java meets OpenCV Motivation - Java 07/11/2013 10 Introduction to OpenCV 

Some facts… 07/11/2013 Introduction to OpenCV 11  The OpenCV APIs have few capabilities for user interfaces  i.e. you can open a window with an image inside. Do you want a button? You cannot have it!  to have a “decent” user interface you need to install “something else” (e.g., Qt for C++)…  … and then compile OpenCV for adding Qt support  C++ learning curve is quite… steep  source: experiences from teaching this course in the last years 

Some (other) facts… 07/11/2013 Introduction to OpenCV 12  OpenCV C++ APIs exists since OpenCV 1.0  It is possible, but not recommended, to “mix” different OpenCV version  Typically, it is a good way to get into trouble!  It is possible to use OpenCV C APIs together with C++ APIs  Typically it is a good way to get into trouble! 

Some (other) facts… 07/11/2013 Introduction to OpenCV 13  OpenCV C++ API exists since OpenCV 1.0  It is possible, but not recommended, to “mix” different OpenCV version  Typically, it is a good way to get into trouble!  It is possible to use OpenCV C APIs together with C++ APIs  Typically it is a good way to get into trouble! Yeah, some students did it! 

A better world is possible? 07/11/2013 Introduction to OpenCV 14  OpenCV provides also the Java APIs  nothing to compile (on Windows)  they works on Android, also  Java 7 has a first-grade user interface  not Swing, but JavaFX  http://www.oracle.com/technetwork/java/javafx  Almost everybody here should have some (basic) experience with Java  Performance between the C++ APIs and the Java APIs are comparable 

A better world is possible? 07/11/2013 Introduction to OpenCV 15  The OpenCV Java APIs are almost identical to the C++ version  knowledge can be transferred!  examples:  to store an image in memory both use the Mat object  to write an image on disk both use the imwrite method  Something change a bit  examples:  CV_RGB2GRAY (C++) becomes COLOR_RGB2GRAY (Java)  Point, Point2d, Point2f (C++) becomes 3 overloaded constructor of Point (Java) 

A better world is possible? 07/11/2013 Introduction to OpenCV 16  Question:  is OpenCV with Java/JavaFX the “best” solution for every application? 

A better world is possible? 07/11/2013 Introduction to OpenCV 17  Question:  is OpenCV with Java/JavaFX the “best” solution for every application?  Response:  No, obviously  Do you need a GUI? Go with JavaFX!  Do you have memory constraint? Go with C/C++!  …  Please, no extremism! 

Getting started with OpenCV and Java OpenCV meets Java 07/11/2013 18 Introduction to OpenCV 

Modules 07/11/2013 Introduction to OpenCV 19  core  basic structures and algorithms  imgproc  image processing algorithms (such as image filtering, geometrical image transformations, histograms, etc.)  video  video analysis (such as motion estimation and object tracking) OpenCV has a modular structure: the package includes several shared or static libraries 

Modules 07/11/2013 Introduction to OpenCV 20  highgui  basic operation to read/write/encode images; in C, C++ and Python it provides also basic UI capabilities  calib3d  camera calibration and 3D reconstruction  features2d  2D features framework (feature detectors, descriptors, and descriptor matchers)  objdetect  detection of objects and other items (e.g., faces, eyes, mugs, people, …) 

Modules 07/11/2013 Introduction to OpenCV 21  ml  machine learning classes used for statistical classification, regression and clustering of data  gpu  GPU-accelerated algorithms  photo  computational photography  ccl  OpenCL-accelerated algorithms 

Data Structures 07/11/2013 Introduction to OpenCV 22  We speak about Java API  All the OpenCV classes and methods are placed into the org.opencv.* packages  Mat  the primary image structure in OpenCV 2.x  overcomes the “old” IplImage/CvMat problems (OpenCV 1.x/C API)  automatic memory management (more or less in C++)  two data parts:  matrix header (contains information about the matrix)  a pointer to the matrix containing the pixel values 

Data Structures 07/11/2013 Introduction to OpenCV 23  Point  2D point  defined by x, y coordinates  Point first = new Point(2, 3);  Size  2D size structure  specify the size (width and height) of an image or rectangle  Rect  2D rectangle object 

Basic Image I/O 07/11/2013 Introduction to OpenCV 24  Highgui.imread  loads an image from file and return the corresponding Mat object Mat Highgui.imread(String filename, int flags)  Highui.imwrite  save an image on disk bool Highgui.imwrite(String filename, Mat img, MatOfInt params) 

Basic Drawing Operations 07/11/2013 Introduction to OpenCV 25  Core.circle  draws a simple or filled circle with a given center and radius on a given image  Core.line  draws a line between two point in the given image  Core.ellipse  draws an ellipse outline, a filled ellipse, an elliptic arc, a filled ellipse sector, …  Core.rectangle  draws a rectangle outline or a filled rectangle  note that negative thickness will fill the rectangle 

Color Spaces 07/11/2013 Introduction to OpenCV 26  Imgproc.cvtColor  converts an input image from one color space to another  examples:  cvtColor(src, dest, Imgproc.COLOR_RGB2GRAY);  cvtColor(src, dest, Imgproc.COLOR_HSV2BGR);  cvtColor(src, dest, Imgproc.COLOR_RGB2BGR);  Important! Images in OpenCV uses BGR instead of RGB 

Getting started (without going crazy) Let’s get practical! 07/11/2013 27 Introduction to OpenCV 

How can we use OpenCV? 07/11/2013 Introduction to OpenCV 28  LABINF:  already installed under Windows  version 2.4.6  Eclipse (Kepler) is the IDE  At home:  you can use whatever IDE you like  but we give full support only for Eclipse  Installation:  see the PDF document in the teaching portal 

What if I got problems? 07/11/2013 Introduction to OpenCV 29  Small problems  drop me a line  [email protected]  Normal problems  come to office hours  every Wednesday, 9:00 - 11:00  LAB6, second floor of DAUIN  please send an e-mail beforehand  Enormous problems  pray?  no, seriously, we can schedule an extra “lesson” Awesome student to me Hi, […] I’m using “cvtColor(image, gray, COLOR_BGR2GRAY);” but it give this exception: […] Can you help me? Regards, … Problems with JavaFX and a gray scale image 

What if I got problems? 07/11/2013 Introduction to OpenCV 30  Small problems  drop me a line  [email protected]  Normal problems  come to office hours  every Wednesday, 9:00 - 11:00  LAB6, second floor of DAUIN  please send an e-mail beforehand  Enormous problems  pray?  no, seriously, we can schedule an extra “lesson” Not-So-Awesome student to me Hi, […] I followed the guide for installing OpenCV on my Mac but I have an error after step 3. Can we meet on next Wednesday to solve the problem? Thanks! Regards, … OpenCV installation 

What if I got problems? 07/11/2013 Introduction to OpenCV 31  Small problems  drop me a line  [email protected]  Normal problems  come to office hours  every Wednesday, 9:00 - 11:00  LAB6, second floor of DAUIN  please send an e-mail beforehand  Enormous problems  pray?  no, seriously, we can schedule an extra “lesson” Good student to me Hi, […] I see the solution of Exercise 2.1 but I don’t understand the following expressions: - main(); - System.out.println(); - @Override. Can you explain to me what they are? Regards, … Help with OpenCV 

An e-mail not to be sent! 07/11/2013 Introduction to OpenCV 32 

Useful Resources… 07/11/2013 Introduction to OpenCV 33  OpenCV Wiki  http://code.opencv.org/projects/opencv/wiki  OpenCV Official Documentation  http://docs.opencv.org/  User Q&A forum  http://answers.opencv.org/questions/  OpenCV Javadocs  http://docs.opencv.org/java/  JavaFX Documentation  http://www.oracle.com/technetwork/java/javafx/document ation 

Put everything together Demo Hour 07/11/2013 34 Introduction to OpenCV 

License 07/11/2013 Introduction to OpenCV 35  This work is licensed under the Creative Commons “Attribution- NonCommercial-ShareAlike Unported (CC BY-NC-SA 3,0)” License.  You are free:  to Share - to copy, distribute and transmit the work  to Remix - to adapt the work  Under the following conditions:  Attribution - You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).  Noncommercial - You may not use this work for commercial purposes.  Share Alike - If you alter, transform, or build upon this work, you may distribute the resulting work only under the same or similar license to this one.  To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/