Did you know the very first Augmented Reality experience was created in 1968? It wasn’t a fancy headset or smartphone app, but a simple head-mounted display used by an architect to visualize his designs superimposed on real-world blueprints. From humble beginnings, AR technology has evolved into a powerful tool capable of transforming our perception of the world around us. In this article, we’ll delve deeper into the fascinating technology behind Augmented Reality and explore how it works its magic.
What is Augmented Reality?
So, what is AR? It’s not so easy to define “augmented reality” because the user experience is fundamentally different from the technical concept it represents. In its global sense, AR technology can be explained as the view of the physical environment complemented with computer-generated visual and/or sound elements.
Brief History of AR
The first augmented reality concepts emerged in 1901 – the year when L. Frank Baum’s novel “The Master Key” has seen the world. Though AR was not called exactly this way, the glasses that the demon gave to the protagonist allowed him to see the letters on people’s foreheads designating their nature (e.g., E for evil, G for good, W for wisdom, etc.). In 1974, the AR lab “Videoplace” was set. However, the augmented reality features were officially documented only in 1990 by Tom Caudell and David Mizell. The first fully functional AR device was presented by Julie Martin in 1994. It was called “Dancing in Cyberspace” for a reason: it was an AR theater that combined virtual objects with real-life environments.
In the 21st century, AR has been developing at a quick pace. For instance, in 2000 Hirokazu Kato with ARToolKit launched an open-open source AR library. It represented a cutting-edge combination of virtual graphics with real-time video tracking for graphics overlapping. Nine years later, ARToolkit got seamlessly integrated into web browsers. In 2013, Google launched the “Google Glass” project, and the world was conquered by the AR game “Pokemon GO” in 2016. 2017 was ushered with the emergence of eye-tracking AR technology. Since 2018, 1,000+ apps have been added to the Google and Apple stores, offering a variety of AR features.
AR vs VR: What’s the Difference?
Augmented reality is sometimes confused with virtual reality (VR), a technology immersing the user inside a synthetic environment and detaching him or her from the real world. The difference between AR and VR is that the former is enhanced reality, not its substitution. In such a way, AR presupposes the overlay of digital, computer-generated information (e.g., video, sound, or animation) over the real-time environment to complement it.
To understand what is augmented reality technology, let’s consider several examples of both VR and AR products.
AR apps include:
- Ikea’s AR app allows customers to choose the right product that will fit their rooms perfectly. They can view the room through the camera on their smartphone. By swiping and clicking Ikea merchandise, they can immediately see if a piece of furniture suits the design in their apartment.
- Gap has recently launched AR-enhanced changing rooms in which consumers can try on a variety of items without wandering across the shop, carrying tons of things.
- Bayern Monaco has launched an AR app that allows users to take selfies with their idols without traveling to Munich and trying to catch the superstars.
- Land Rover’s AR app allows drivers to see the car’s engine through the transparent bonnet.
VR examples include:
- Marriott’s VR “Transporters” app allows visitors to attend its hotels in different parts of the globe. The application is highly realistic and can bring in a genuine travel experience.
- The VR simulator of SportX lets the customers test the sportswear it sells in a variety of simulated scenarios, such as running, swimming, or playing tennis.
- The HoloTeach VR app allows surgeons to interact and consult with each other in real time during surgery, even if users are thousands of miles away from each other.
AR represents a type of interactive reality that integrates the computer-generated elements of video, sound, or graphics/animation into reality to create an enriched user experience.
Types of Augmented Reality
The modern augmented technology comes in different kinds due to the recent developments in the IT industry. Here is a brief breakdown of the main types of augmented reality popular with businesses and users today.
Marker-based AR
At the dawn of AR development, a marker (a thick black square border) was used as a reference object to measure the distance from a computer to the object. In other words, the marker-based AR app relies on the reader (which is most often a camera) to read an image and produce 3D objects in the virtual environment.
Markerless AR
Despite some evident benefits of marker-based AR, the visual appearance of the thick black marker was not appealing, which reduced the popularity of marker-based solutions in the market. For commercial use, the markerless AR was specifically developed to overcome the pitfalls of the previous version. In this AR type, general images serve as markers, which are availed by image extraction technology, such as SIFT, SURF, and FAST-SURF.
Most modern markerless AR systems utilize the robust point matching approach enabling dealing with the problem of robustness due to outliers, large occlusion, etc. With its help, you can detect and map the real-world environment, placing virtual objects into it. Examples of markerless AR include anything, from a dancing animated creature on the table to the IKEA sofa placed into your interior.
Projection-based AR
As its name suggests, the projection-based type of AR employs a video projector for the display of images on the screen or a variety of physical surfaces. At the basis of this AR type lies the use of real-world objects for the projection of virtual images. It is commonly used in industrial assembly and product visualization. However, portable projection-based AR is limited to a certain extent in terms of the quality of projection on heterogeneous, oddly shaped surfaces because of the differences in reflectance, color, and geometry.
Superimposition-based AR
This type of augmented reality presupposes the superimposition of the graphical additions to the real object right on it (or its fully reproduced image), which means giving an augmented view of the real object. The ability of technology to recognize objects plays a critical role in superimposition-based AR execution since the inability to identify an object leads to the impossibility of replacing the original view with an augmented one. An illustrative example of augmented reality of this kind is the real-time medical examination of patients. For instance, some medical institutions provide a live feed from an X-ray machine to superimpose the X-ray results on the real image of the patient’s body.
How Does Augmented Reality Work?
AR applications are no longer limited to entertainment only. This technology serves profound business, scientific, and research purposes. Here are some details about core techniques and tools used to create immersive AR experiences.
Sensors and Cameras
The creation of any AR elements requires precise capturing of real-world objects to augment them realistically on your display. For these purposes, a variety of sensors has been adopted in the AR software, such as mechanical, biological, acoustic, optical, and environmental ones.
Mechanical sensors capture the object’s position, acceleration, shape, mass, and displacement. Their common areas of application include the determination of the object’s position, weight, and movements.
Biological sensors measure the object’s heart rate, temperature, neural activity, and respiration rate. They are applicable in the area of mood identification, as well as the evaluation of mental and physical states of people.
Acoustic sensors provide accurate data on the sound’s volume, pitch, frequency, phase, and modulations, which is highly valuable in terms of sound detection and speech recognition.
Optimal sensors report data on the object’s emissivity, refraction, light wave frequency, brightness, and luminance, which is handy in terms of computer vision detection, presence detection, and IR motion analysis.
Environmental sensors provide data on temperature and humidity of the surrounding environment.
One of the crucial tools to build fascinating AR experiences is the depth-sensing camera – a camera that can capture 3D images. AR devices traditionally have at least one camera, while some of them have two for depth sensing. It’s also common to install an IR camera in the device to allow heat mapping and advanced depth sensing.
Projection
Projection is an AR feature that provides the possibility of augmenting real-world objects to wear or carry any devices. It means that the augmented reality elements are projected directly on the objects via the spatial AR tools, using controllable projector-camera-systems. Projection AR systems are a cutting-edge technologies in the area of augmented reality since they offer completely unencumbered AR experiences, enabling users to enjoy projection on the go by displaying the augmented reality elements on the environment.
Processing
AR is essentially made possible due to the sophisticated image and sound processing techniques. Any AR product relies on image processing for the sake of creating precise environmental understanding, light estimation, and determination of critical feature points and planes.
Reflection
The basic goal of any AR technology is to generate photorealistic images via the integration of virtual objects into real scenes. Realistic images can be created when a developer uses advanced modeling to connect virtual and real evironments. Virtual reflection is the technique used in AR to simulate reflections of real-world objects on virtual objects augmenting the real-life scenes.
What Devices Does Augmented Reality Work On?
After you have the main principles of how augmented reality works, you may be naturally interested in exploring a more practical side of AR technologies, that is, the devices used for its production.
Mobile Devices
Though only a couple of decades ago, AR could be experienced only in specially equipped rooms, which required much-specialized equipment, today’s mobile apps deliver realistic AR experiences. Nowadays, one can enjoy AR on both iOS and Android devices. Some popular apps include the world-famous game Pokemon Go as well as the Froggipedia app, BBC Civilizations AR, and SketchAR. Popular messengers and social networks also let us explore the power of AR (e.g., Hololens, Snapchat filters, etc.).
AR-based Devices
While the mobile tech is developing rapidly bringing new inventions to the world, more realistic, advanced, and professional-looking AR is still possible to experience only with the help of special AR devices, e.g. wearable smart glasses, head-mounted apparatuses, gyroscopes, digital compasses, GPSs, CPUs, and displays. The most common special device is the head-up display (HUD) that a user puts on to acquire a fully immersive AR experience.
AR Glasses
Smart glasses have also gained much popularity among such popular brands including Google Glasses and Laster SeeThru. AR-enriched lenses have also been incorporated into light-weight AR glasses, e.g. the AV Walker – the Japanese AR glasses overlaying the world with digital content.
AR Contact Lenses
Incorporating the AR technology into contact lenses is a relatively new practice in the tech industry.The pioneering project named “Twinkle in the Eye” was launched by the University of Washington to create the AR lenses that will be able to display texts, translate speech into captions, and provide visual clues from the navigation system in real time.
Virtual Retina Display (VRD)
The VRD emerged in the late 1990s as a cutting-edge technology for creating visual images.The first one was developed at the Human Interface Technology Laboratory (HIT Lab). The method of image creation via the VRD is based on lower-power laser light scanning and image reproducing directly onto the human retina. The use of the VRD allows high-resolution, high-contrast image generation, with images superimposed over whatever its users see.
How is Augmented Reality Controlled?
Currently, the functionality of AR apps can be controlled with the help of features and functions developed by engineers. A user chooses how to behave, which scenario to follow and when the game should come to the end. However, it is expected that new AR apps will be managed with the help of the human mind. Though it is hard to envision today, such control is absolutely possible to come in the next few years. Such startups as Neurable take the first steps to make it possible for a user to control VR & AR apps with a “brain mouse.”
The Future of Augmented Reality
With technology evolving at an unprecedented pace, we can expect AR to advance quickly, delivering brand-new experiences for users worldwide. So, what does the future hold for augmented reality users?
- AR Contact Lenses. While the prototypes of AR lenses are already in use today, their mass market entry is a matter of future. Soon we can expect to have wireless lenses with hundreds of LEDs shaping images in front of the human eye. These images may include words, charts, and photographs, dynamic instructions, and entertaining AR elements.
- Bionic Eyes. With biomimicry and bionics developing at a fast pace as well, the humanity can expect the introduction of the AR technology in this area. For instance, the bionic eyes may help blind or people with poor eyesight to restore their vision.
Conclusion
As you can see, AR is a promising area of technology providing enriched visual experiences and changing the ways in which learning, navigation, medical interventions, and communication are conducted. AR devices are becoming even more sophisticated, consumer-friendly, and lightweight, thus bringing the AR experiences to regular users and letting them have fun, acquire new knowledge, travel to distant countries, and meet people thousands of miles away in one room.
