SteerBy

Introduction

SteerBy is an indoor navigation application powered by Augmented Reality (AR) technology, developed as the final project for the Mixed and Augmented Reality course (CS291A). The primary goal of this project is to guide users to specific destinations within a building, such as rooms, tables, or other points of interest. The app utilizes AR to overlay a user-friendly interface on top of the real-world environment, providing seamless navigation and enhancing user experience. By addressing the challenges associated with traditional indoor navigation methods, SteerBy aims to revolutionize the way users navigate indoor spaces.

Problem Statement

Indoor navigation has long relied on traditional methods such as physical maps, static signs, and verbal directions. However, these approaches often fail to provide a seamless and accurate navigation experience for users. There is a clear demand for an intuitive, digital solution to address the challenges of indoor navigation.

Augmented Reality (AR) has the potential to revolutionize indoor navigation, but several obstacles must be overcome to develop a viable AR-based solution. Some of the key challenges are as follows:

1. Lack of a comprehensive indoor "points of interest" entity database: An effective AR navigation system requires an extensive and up-to-date database of indoor points of interest, along with their attributes and AR Anchor persistence. Creating and maintaining such a database is a significant technological challenge.
2. Privacy concerns in non-public spaces: Offices, private buildings, and other non-public spaces have inherent privacy concerns that make it difficult for large technology companies to access and map these environments. This limitation complicates the development of a universally applicable AR navigation solution.

Addressing these challenges is crucial for creating an innovative and practical AR-powered indoor navigation system that can guide users to their destinations quickly and accurately.

Solution: SteerBy

SteerBy Demo [Marine Sciences Hall @ UCSB]

SteerBy Demo [Phelps Hall @ UCSB]

Multi-floor support [Phelps Hall @ UCSB]

Project Overview

Objectives

The primary objectives of the SteerBy app are as follows:

1. To develop an AR-powered indoor navigation system that accurately guides users to their desired locations.
2. To provide an interactive and user-friendly interface that simplifies the process of adding and selecting destinations.
3. To explore and implement advanced AR techniques for an improved user experience.

Features Implemented

The following features have been implemented in the SteerBy app:

1. ARWorldMap Serialization and Deserialization: The app utilizes ARWorldMap to store the state of the user's environment, including the detected real-world features and virtual content placed by the app. The world map is serialized using NSKeyedArchiver and deserialized using NSKeyedUnarchiver to save and restore the session state.
2. ARWorldTracking: The app uses ARWorldTrackingConfiguration to track the device's position relative to objects in the environment. Ray-casting is employed to find real-world surfaces corresponding to a point in the camera image.
3. Focus Entity: A supporting package, Focus Entity, provides user feedback on the camera focus and assists in finding the appropriate position to place anchors in the physical world.
4. Anchor Orientation with StickyNoteEntity: StickyNoteEntity is used to place virtual sticky notes with text in the real world to identify points of interest. The text displayed on screen is readable at all viewing angles and distances.
5. Arrow Pointer to AR Anchor: The navigation component, RKPointPin, is a UIView component that sticks to an ARView and points to the chosen anchor within the scene.

App Screenshots

Augmented Reality Techniques Explored for Future Implementation

• ARKit Occlusion: The app plans to implement occlusion techniques to improve the visual accuracy of the AR experience. This would involve informing the app about the location of walls and real-world geometry and setting up appropriate masking planes and physical bodies.

Personal Assessment

Achievements

The primary goal of designing an AR-powered indoor navigation app has been successfully achieved within the project timeline. The app can now:

1. Request and obtain user permissions for camera and location access.
2. Add anchors to the world map, save them to a session, and load them in a different session.
3. Display a list of anchors in the world for the user to conveniently select a destination.
4. Guide the user to the destination anchor with the help of an arrow pointer.
5. Exit the navigation upon clicking the "Exit" button.

The major challenge of integrating different packages and frameworks was overcome through continuous small-scale testing throughout the development process.

Limitations

One limitation of the current project is the integration with the backend system. Further work is needed to ensure seamless communication between the app and the server for efficient data storage and retrieval.

Future Scope

SteerBy has immense potential for expansion. The ultimate vision is to create a scalable, markerless AR navigation solution that can be enriched by users, similar to how OpenStreetMap and Waze revolutionized GPS-powered street navigation.

References

1. Research Paper: Towards An Indoor Navigation System Using Monocular Visual SLAM
2. Research Paper: A Multi-Scale Representation of Point-of-Interest (POI) Features in Indoor Environments
3. Apple Developer Documentation: ARKit
4. GitHub Repository: FocusEntity
5. GitHub Repository: StickyNoteEntity
6. GitHub Repository: RKPointPin