A student team at Carnegie Mellon University’s Entertainment Technology Center (ETC) in Silicon Valley has created an augmented reality (AR) application that features a 3D virtual assistant that interacts with the user and the real-world environment. The application uses the Epson Moverio BT-200 smart glasses to place a virtual assistant named “Sparky” in the user’s field of view to assist with real-time navigation.

Most current augmented reality technology applications use markers or computer vision to drop static objects or pre-produced animation into the environment. We wanted to explore creating a more natural experience through interacting with a character in the real world. The Moverio BT-200 smart glasses allowed us to bring these augmented reality characters to life with their binocular transparent display, gyroscope and other features.
— John Shields, Carnegie Mellon student

Taking advantage of stereo vision to give the character a sense of depth and reality in the their field of view, users can interact with the character using voice and gestures and have meaningful and fun interactions based on its knowledge of the real-world environment.

The team did an excellent job in creating an augmented reality navigation assistant that also acts as an interactive companion to entertain and engage users. The application offers a glimpse of the future of augmented reality’s role in daily human life.
— Carl Rosendahl, Professor at Carnegie Mellon University