Augmented reality (AR) – How this emerging technology can augment the mobile experience

Augmented Reality (AR) is where digital images and characters can be imposed onto the real world through a phone’s screen and have the potential to be meaningful and useful for mobile usage.

Current AR frameworks not only allow app developers to place digital elements onto the real world, they can also bring elements from the real world and place them into the digital world.

By giving phones the ability to determine their position and orientation in a real-world environment (a phone knows where it is and what it is looking at), it gives phones the ability to identify surfaces such as tabletops, countertops, floors, rugs, walls, ceilings, and possibly the palm of a hand, in the real-world environment, and place AR objects including lighting effects on those surfaces.

How to Make AR an Indispensable Mobile Experience

First, don’t just retrofit AR onto existing apps; it takes planning and an integrated approach to create a smooth interaction. AR interactions need to be slick, useful, and feel natural, and they need to work well in an app when they are well integrated into the greater app workflow.

• AR specific affordances should be provided to guide people without overwhelming them
• Don’t overload the user with too many new choices
• Slow and progressive familiarization, visual clues, and guidance from the app should be used to help the user
• In designing for AR, designers should leverage size, contrast (including lighting effects) and texture/color to denote hierarchy
• In AR, size is the most important cue; it’s based on the distance between the phone and the surfaces in the environment, and the objects of interest, so it’s important for the app to understand the scale of the environment and place AR objects/characters to fit the viewing distance in the phone

AR interactions (ideally) need to become habitual, performed on a daily (if not hourly) basis.

• To attract the daily attention of the consumer, AR constantly needs to jab people
• AR is a great form of guerrilla or ambient media interactions, adding value in unexpected situations
• AR needs to be smart about personalization to attract and keep daily attention, which will eventually help create desire – how did I live without this!
• Once people decide an AR interaction is indispensable to daily life, they’ll impulsively use it

AR interactions need to be monetizable and/or contribute to a brand halo.

• Customer usage data
• In-app purchases
• Subscriptions
• In-App advertising

AR can add fun to the Mobile Experience

Standing on the shoulders of Pokémon Go, Snap’s AR filters advanced the fun 3D character experience with animated 3D characters that can be placed into the scene by the user. The placed Snapchat AR character detects the boundaries of objects and surfaces on the screen.

At the past two Worldwide Developers Conferences (WWDC), Apple showed continued commitment to AR by announcing ARKit 2, alongside iOS 12. In addition, Apple will partner with Pixar and Adobe to create a new AR format, to be named the Universal Scene Description file (USDZ). The USDZ format is a zero compression, unencrypted zip archive supported across iOS, so whereever you see an AR icon in iOS, it means that the object is an AR object, and can be experienced in an identical way across iOS apps. Adobe also announced it will have native USDZ support in Creative Cloud suite, allowing designers to build and lay out 3D objects more effectively.

Potentially Useful AR Mobile Interactions

There are numerous mobile usage scenarios where well-designed AR has the potential to improve workflow and experience. The following examples represent applications both not yet widely seen but showing good potential for wide usage. The majority of these fall in the category of HUD (heads-up displays) functionality within apps.

(1) Providing Directions in Unfamiliar Environments

The following example is a mall app that leverages AR to leave you AR breadcrumbs (your path) to help guide you back to your car in a giant mall parking lot. It is also useful for stadiums, convention centers, outdoor concert venues, and other crowded areas.

The following example is a hospital app that could take you to the room you are visiting or to your own appointment – just follow your own line on the floor. AR signage even has the potential to eliminate multilingual physical signage.

(2) Hospitality Aids in Unfamiliar Cityscapes

This example is a tourism app that all but makes physical signage unnecessary and can also augment the signage in useful ways by essentially creating multilingual physical signage through the phone screen.

(3) Providing Organization to a Chaotic Day – Enhancing the Convention Experience

AR could transform the experience of attending a convention and elevate it into something far more productive and less stressful. At AR “enabled” locations throughout a convention or conference hall, you can have AR convention guides greet customers, giving them instructions on schedule, events, seminars, demos, catering, hospitality, etc. And with AR paths on the convention floor, an attendee will be able to find a specific session or trade booth far more easily.

The following example is convention center app showing an AR character that provides a guide to presentations along with presenter info; when you pick one, you can follow the AR path to the room.

(4) Previewing Body Art or Cosmetic Procedures

AR could transform the experience of selecting and placing body art or visualizing the benefits of cosmetic procedures by being able to visualize the effects in 3D.

The following example is a body art app that would let you select a tattoo, and with another person pointing the phone at you, the tat is “placed” on your body to help you preview its appearance. The placing is realistic, taking into consideration the contours of the body.

(5) Enhancing the In-Stadium Experience: Roleplay coach while watching sports in real time

Allow in-venue sports fans to be more “involved” and “immersed” in the action on the field in front of them by predicting plays on the phone and seeing if they are right. Sports fans can also engage in sports betting by calling a play that is about to happen.

The following example shows a fan in his seat at an NFL stadium, pointing the phone at the play on the field and picking what he thinks will the next offensive formation. The AR positions and routes appear under players on the field, and as the play starts on the field, the AR positions move with the players. If the actual play matches the prediction, the fan is rewarded.

(6) Wedding Planning

Helps people envision their wedding venue and try out the desired layout.

This could be done by walking around a wedding venue and placing 3D models of various wedding related items (including the cake) on the phone screen to see how they look or to discover their ideal placement.

The following is a wedding planning app screen that shows an AR wedding cake on a dressed table, placed in the center of a wedding hall at which the phone is pointing. The screen also shows AR models of the cake’s and table’s placement, which can be manipulated by the user.

(7) Data Visualization

3D AR data visualization can be used on the phone to help envision problems and/or opportunities by overlaying data over a real-world environment.

This is a sample of a real estate app screen that displays AR data sets placed throughout a city view on a phone screen, relying on images taken by a drone. The 3D AR data shows approximate building height based on available air rights.

And the following is a crime statistics app showing locations of crimes, placed throughout a city view, on a phone screen, relying on images taken by a drone.

Another version of this scenario is of a data visualization app for meetings and presentations showing AR 3D data models hovering on top of paper charts.

To Wrap Up

AR interactions in apps need to be slick, useful, and feel natural in the environment as seen through the screen. As seen in the initial apps Apple highlighted to showcase AR in iOS11, smooth AR interactions that feel integral to the workflow of the app will add the most value and will be embraced.

Really clear affordances should be provided to guide people in applying AR interactions without overwhelming people. In designing for AR, designers should leverage size, contrast (including lighting effects) and texture/color to denote hierarchy, but size is the most important cue.
AR app features will succeed if there’s something short, useful, and memorable that grabs the app user and it’s something that grabs people in a way that makes them want more.

Get in touch with us to find out how Atimi Software can help you build a custom, innovative, enterprise app that offers a superior AR user experience.