Geographical Information Systems & Location-based Augmented Reality A Review of Techniques, Challenges, and Possibilities Johnny Luce Design Director and co-founder, augzoo LLC.

Johnny Luce Projects Employers Universities

Today’s Agenda Define Augmented Reality (AR), with emphasis on location-based AR Describe various techniques used to achieve location-based AR Look at considerations for creating and storing content for location-based AR Explore user experience and possible use cases location-based AR

A technology that superimposes a computer-generated image on a user's view of the real world, thus providing a composite view. Augmented Reality

Augmented Reality Three forms of Augmented Reality Marker-based Markerless Location-based

Marker required to activate Marker-based Augmented Reality Augmentation anchored to marker Markers: Image or QR code Easiest form of augmented reality Affected by lighting conditions Strict image requirements

Markerless Augmented Reality Simultaneous Localization and Mapping (SLAM) Points used to calculate distance and angles Uses Computer Vision to align content to surfaces Computationally intense Affected by lighting conditions Error prone

Markerless Augmented Reality Simultaneous Localization and Mapping (SLAM) Points used to calculate distance and angles Uses Computer Vision to align content to surfaces Computationally intense Affected by lighting conditions Error prone

Location-based Augmented Reality Provides full 360° experiences Most computationally intense Uses GPS + SLAM plus + metadata Can be independent of lighting Very error prone

The Challenge + = Accurately align virtual objects with real-world locations to produce convincing spatial interactions.

Location / Position GPS 4 meter accuracy WiFi and RFID improves accuracy Latitude and Longitude Affected by buildings and weather

Elevation and Altitude Altitude: height above ground Match eye level Elevation: ground height Match floor of building Hard to measure

Heading Measured from magnetic North Many accuracy issues Forward facing direction Subject to Gimbal lock

Pose Measured in Quaternions Difficult to measure accurately Roll, Pitch, and Yaw of the device Computationally intense Most important for believable 3D environments

Motion Tracking Tracks unique points over time Points used to calculate angle and distance Computer Vision technique Computationally intense Affected by lighting conditions

Occlusion Culling Adds depth and realism Very hard to calculate Disables rendering of objects when they are not currently seen by the camera

Motion Tracking: Occlusion

Collision Decection Adds depth and realism Can be very hard to calculate The computational problem of detecting the intersection of two or more objects

Motion Tracking: Collision Collision

The Challenge: Reprise + = Accurately align virtual objects with real-world locations to produce convincing spatial interactions.

Sensor Fusion Goal Hardware Data Location / Position Global positioning System, Wifi, RFID GIS and CAD files Elevation and Altitude Barometer, Wifi, RFID Markers, GIS and CAD files Pose / Heading Magnetometer, Camera, Infrared, Gyroscope, Accelerometer Markers, GIS and CAD files Occlusion and Collision Camera,Infrared GIS and CAD files The process of merging data from multiple sensors such that to reduce the amount of uncertainty.

Primary GIS Data Types Elevation Vectors Data: GIS and CAD

Data: GIS and CAD

Data: GIS and CAD

Data: GIS and CAD

Data: GIS and CAD

Cloud-based data storage

Cloud-based data storage Table from a database. One object per row, with columns defining each object’s properties.

Data: GIS and CAD Static Data Raster Vector Mesh Table from a database. One object per row, with columns defining each object’s properties.

Data: Raster Stores elevation and or color values Each Pixel represents area, or resolution Images: GeoTIFF and MrSID, et al. Requires lots of disk space

Data: Vector Points are defined by X,Y or Latitude, Longitude Points can be ordered as lines Stored in database or file: .KML. .SVG, .JSON Lines can be closed to form areas Good for features and locations Metadata to define features

Data: Mesh Often has images for color: dtx, png, tiff... Cartesian position, rotation, and scale Hand-crafted. Many formats: .obj, fbx, kmz... Provide high quality content Expensive and time consuming to produce Computationally intense

User Experience Don’t do AR just because it is novel. Use appropriately Make tasks simple, clear, meaningful Environmental and spatial interactions are not always intuitive

User Experience

User Experience THIS AUGMENTED REALITY FUTURE LOOKS LIKE A LIVING HELL

Thank you