As the hairdresser holds up a tablet computer to show you your ‘reflection’, you flick your head to see the virtual bounce and fall of your next hairstyle. “Looks great,” you say. “Just leave it a little longer at the back.” With a swipe, he perfects your new look on-screen – then picks up his scissors and starts to snip.
Welcome to augmented reality – the technology that enhances our view of the world by blending it with computer-generated images.
Fighter pilots have used it for two decades, projecting information on to their cockpit windscreen to help them identify targets. Now it’s reaching the mainstream. Augmented reality apps, that add digital objects to the view taken by your smartphone or tablet camera, can already be downloaded. But the biggest buzz surrounds the imminent arrival of augmented reality headsets.
The technology is different from virtual reality headsets. Virtual reality uses tiny computer screens placed close to your eyes to immerse you in a computer-generated world.
With augmented reality, you still see the real world – but with some digital embellishments. Microsoft’s prototype augmented reality headset, the HoloLens, doesn’t even use a screen. The device adjusts the scene before you by projecting light directly on to your retina, conjuring virtual objects that Microsoft calls holograms.
These holograms are not idle phantoms. Sensors borrowed from Microsoft’s Kinect platform allow you to manipulate the holograms with your hands, like Tom Cruise does in the movie Minority Report. The holograms can also interact with real objects – if you ‘pick up’ a virtual ball and drop it on to a real table, it will (virtually) bounce.
Gamers are likely to be the first to use the technology – imagine blasting virtual zombies crawling across your living room floor.
The potential applications are mind-boggling. NASA, for one, is taking notice. In December 2015, prototype HoloLens hardware was sent up to the International Space Station. Cameras built into the headset mean that by looking at their computer screens the ground crew can see what the astronauts are seeing. What’s new is that ground operators can now draw virtual annotations on to that view. These appear in the astronaut’s vision in real time, to guide them through difficult procedures, for instance. So an arrow might rise up before them when they are connecting a wire, first pointing to the wire and then the connection point.
Back on Earth, gamers are likely to be the first to use the technology – imagine blasting virtual zombies crawling across your living room floor.
HoloLens is not perfect. The light projection technology can only conjure objects in the centre of your vision, and the images it creates are small – about the size of a smartphone held at arms length. If a digital object is any larger, you can only see a part of it at a time.
But Magic Leap, a tech startup working on a rival device with a bigger field of view, has already raised $540 million from investors including Google.
HoloLens will probably reach the market first. Game and app developers will get their hands on the hardware in early 2016, with consumer models expected a year or so after that.
In the meantime, we can have an early taste of augmented reality by downloading apps on our smartphones and tablets. These can add digital elements to the camera view. For example, Google Translate can already transform foreign roadsigns into English before your eyes.
And Disney is working on a new generation pop-up book. By looking through a tablet at a colouring book, for instance, kids will be able to see their drawings stand up in fully rendered 3-D.
'Like your mobile phone it is going to become part of your life.'
Augmented reality could also change how we shop. You could ‘try on’ a virtual jacket while browsing online. Or, when visiting a supermarket, “imagine scanning the shelves with your phone to highlight only ‘Australian made’ with no nuts and low salt”, says Bruce Thomas, a researcher in augmented reality at the University of South Australia.
The potential uses are wildly diverse. There won’t be any one “killer” application of augmented reality that everyone will love, says Thomas, but thousands of little improvements to our world.
“Like your mobile phone, it’s going to become part of your life,” he says.
Nasa: Project sidekick
On board the ISS, the equipment for a new experiment is malfunctioning. Back in Houston, an expert technician can see exactly what the astronaut sees – and identifies the problem. On his screen, he zooms in and marks the series of electrical connections that need to be made.
The colour-coded drawings appear instantly as holograms in the astronaut’s vision. The holographic processing maps the astronaut’s 3-D environment so that he sees the markings in their correct orientation no matter how he turns his head, or which way up his body floats.
HoloLens Hardware
The headset is a self-contained computer with more processing power than most laptops.
Holographic processor unit
A new kind of chip integrates data from the headset’s sensors to build holograms that interact with the real world.
Sensors
Depth sensors map the environment around the wearer; an accelerometer, gyroscope and magnetometer track the wearer’s movement.
Sound
Spatial speakers convey ‘where’ a virtual sound is coming from.
HoloLens specifications
Visual: Holograms appear as high-definition images
Weight: 0.4 kg
Release date (developer edition): Early-mid 2016
Price (developer edition): US$3,000
