Augmented reality (AR) is essentially a means for “editing” the input that we receive. Most commonly, visual data is overlaid on the “real world” as a tool for expanding our experience — a directional arrow to follow, for example. A project from the Imperial College London has moved into an area that has been discussed at length: using a virtual reality headset the system is able to make real-time changes to the appearance of real-world objects.

Video from a RGB-D camera (full color, plus the ability to create a 3D map by measuring contours ahead of it) is fed into an Oculus Rift virtual reality headset, then processed to recognize flat surfaces. These surfaces are then “painted” with digital content — essentially overriding the “real world” with a virtual one. For example, your worn and damaged laminate floor can be digitally upgraded to appear as luxurious carpeting or an otherwise blank wall can be filled with social media content.

Potential uses are both fascinating and terrifying: digital content could be overlaid anywhere it might be useful, imagine a document being placed on a tabletop for review. Alternatively, physical content could be modified before we ever get to experience the “real” thing.

Perhaps the most obvious possibility is in marketing, where options could be presented for the appearance of real-world items, without the need for a store full of similar products. For example, a single real-world table could be seen in an infinite number of materials and colors. This example can be taken even farther: as the AR experience is entirely first-person, anyone observing a single real-world object could have an experience unique to them (i.e. the glossy white lacquer desk that I see could appear as stained cherry-wood to you).

For the hyper-technical: the paper, Dense Planar SLAM, by Renato F. Salas-Moreno, Ben Glocker, Paul H. J. Kelly and Andrew J. Davison, is available as a PDF File (patent pending) — direct link.