M.Tech Seminar Presentation

Transcript

1. Imaging Large Multiple Planar Bounded Regions Presenter: Sona Praneeth Akula – 153050050 Guide: Prof. Sharat Chandran 1
2. Image Mosaicing • Well studied problem in computer vision • Creating a image with large field of view by stitching images having a smaller field of view • Smaller images have overlapping regions 2
3. Figure: Mosaic of a scene created from multiple images of small field view. Image Courtesy: http://www.slideserve.com/howell/csce-641-computer-graphics-image-mosaicing 3
4. Problems with mosaicing? • Details not available. What details? 4 Figure: Details of the dial not clear from the larger image
5. Problems with mosaicing? Figure: Manually capturing the scenes by traversing the camera along a direction 5
6. Problems with mosaicing? • Planar and concave surfaces • What about convex surfaces? • Change in position as well as camera angle 6 Figure: Change in camera angle and position for capturing images from a convex surface
7. Figure: Original photo taken from an elevation. Observe that the wall is curved and details are not available( For Example we cannot read the content from posters) Problems with Mosaicing? 7
8. Figure: Panorama of the scene obtained from smartphone (See red color marked regions for defects and blue colored regions for blending artifacts) Problems with Image Mosaicing 8
9. Quadcopters •Advantages •Autonomous navigation possible •Low-cost •IMU data in indoor environment •Still the problem of multi planar mosaicing is not solved!!! 9
10. What should be done? •Estimate planes corresponding to the regions •Extract the bounded regions from the planes •Cover each surface independently 10
11. Past work •Work [4] has been done to create a panorama for single planar surfaces using quadcopter •Not useful as ideally world is made up of multiple surfaces
12. Data from quadcopters? •Fit multiple planes – require 3d points •Where do I get them? •Use PTAM [6], to give 3d map of environment •Consists of 3d points used for plane fitting
13. What next ? •Estimate the scene as multiple bounded regions •Get the images (orthogonal view) of such regions •My Focus •Estimating a given scene as planes •Extracting bounded regions from the obtained planes
14. Prior Literature Sequential RANSAC • Detects multiple models • Runs RANSAC [1] in iterations • Estimate the model and remove the data corresponding to the estimated model • Repeat until data is completely left with outliers (or) no models can be further estimated • Problems? • Estimating a wrong model initially affects further models
15. Sequential RANSAC output (star11 dataset)
16. Prior Literature MultiRANSAC • Principled approach than sequential RANSAC [2] • At each iteration, M models are instantiated and their corresponding consensus sets are found • Combine new consensus sets with previous ones • consensus sets are updated by calculating the maximum cardinality of the combined consensus set and finding a disjoint set with the combined consensus set • collection of all such sets is the new consensus set • Problems • Fails on intersecting models • Number of models(M) is required (Not known beforehand)
17. MultiRANSAC output (star11 dataset)
18. Prior Literature J-linkage • Robust method to outliers • 3 stages • Randomized Sampling (RANSAC) • Making preference sets • Agglomerative clustering • Uses Jaccard distance for clustering -> metric for degree of overlap
19. Discussions J-linkage •Advantages of J-linkage •Robust to outliers •Takes a global view of data, leading to better plane estimation •Disadvantages of J-linkage •Does not give bounded regions •Number of planes detected is non-deterministic
20. Discussions J-linkage ρ - Probability of picking atleast K outlier free MSS composed only of inliers for a given model M - Number of initial hypotheses p - Probability of drawing a minimal sample set(MSS) of cardinality d composed of only inliers. δ - Fraction of inliers for a given model
21. J-linkage output (star11 dataset)
22. Datasets Simulated Dataset Virtual Environment in gazebo Corresponding 3d points from PTAM
23. Datasets Real World Dataset Auditorium wall Corresponding 3d points from PTAM
24. Our Approach Step 1: Obtaining initial set of planes • Feed the points from PTAM to J-linkage • Get the initial labels and corresponding models defined by • Remove the outliers as proposed by J-linkage Planes estimated from J-linkage clustering for simulated data Planes estimated from J-linkage clustering for real data
25. Our Approach Step 2: Re-clustering pseudo-outliers • How do we do that? • Re-cluster the points using k-means • Remove points far from the plane Pseudo outliers present in simulated data Pseudo outliers present in real world data
26. k-means output Re-clustering of pseudo outliers in simulated dataset Re-clustering of pseudo outliers in real world dataset
27. Our Approach Step 3: Get Bounding regions •Get the projections of 3D points onto the plane •Project the XYZ co-ordinates to UV domain •Get the bounding rectangle in UV domain •Re-project the bounding rectangle corners in XYZ domain •At plane intersections, remove the extra extending part of the plane
28. Bounding rectangles (not continuous) Simulated world in Gazebo Auditorium wall
29. Bounding rectangles (continuous) Simulated world in Gazebo Auditorium wall
30. Results Figure: Bounding boxes shown to users for the simulated world. Green rectangle - user marked area. Colored boxes are estimated regions
31. Results Figure: Bounding boxes shown to users for the auditorium. Green rectangle - user marked area. Colored boxes are estimated regions
32. Summary • Aim • Create mosaic for large multiple planar scenes with details • Problem Statement • Acquiring data • Estimating the scene as multiple planes • Extracting bounded regions • Reviewed • Sequential RANSAC • MultiRANSAC • J-linkage • Succeeded in estimating bounded regions from the scene
33. Future Work • To complete the application • Navigate the quadcopter to each region • Capture the orthographic view of the region in small images • Create a mosaic of the scene using the obtained images • Improvement in J-linkage • Number of planes to be deterministic
34. References(1) [1] M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, 1981. [2] M. Zuliani, C. S. Kenney, and B. Manjunath, “The multiransac algorithm and its application to detect planar homographies,” in Image Processing, 2005. ICIP 2005. IEEE International Conference on, vol. 3. IEEE, 2005, pp. III– 153. [3] R. Toldo and A. Fusiello, “Robust multiple structures estimation with j- linkage,” in Computer Vision–ECCV 2008. Springer, 2008, pp. 537–547. [4] Meghshyam G. Prasad, Sharat Chandran, Michael Brown “Mosaicing scenes with a quadcopter,” in WACV (Workshop on Applications of Computer Vision), 2016 (Yet to be printed)
35. References(2) [5] J. Engel, J. Sturm, and D. Cremers, “Camera-based navigation of a low- cost quadrocopter,”in Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on. IEEE, 2012, pp. 2815–2821. [6] G. Klein and D. Murray, “Parallel tracking and mapping for small ar workspaces,” in Mixed and Augmented Auditorium wallity, 2007. ISMAR 2007. 6th IEEE and ACM International Symposium on. IEEE, 2007, pp. 225– 234 [7] R. Toldo and A. Fusiello, “Auditorium wall-time incremental j-linkage for robust multiple structures estimation,” in International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), vol. 1, no. 2, 2010, p. 6. [8] Jackob Engel. tum_ardrone. http://wiki.ros.org/tum_ardrone, 2014.
36. Thank You