Augmented Reality in Newspapers: Technology, Uses, and Problems

Augmented reality is an interesting feature in newspapers that takes readers beyond the printed page. It enables people to see a video, animation, or other unexpected content that is apparently located on a page of their newspaper. The illusion is created when an augmented reality program and the camera software of a smart phone or tablet work together. The newspaper must be viewed through the camera of the mobile device in order for the illusion to work. In addition, the device must be connected to the Internet. Though the added content is not really present in the paper, the AR feature can be an enjoyable enhancement.

When augmented reality is used with a newspaper, a photo in an article is scanned with the camera of a mobile device. The AR software identifies the photo and then loads related digital content. The content is displayed in the device's camera view. It's often positioned over the scanned photo so that the digital content appears to have replaced the photo.

Although the content that is loaded is frequently a video, there are many other possibilities. A photo gallery, the latest news updates or sport scores, related social media information, educational animations, additional facts, a relevant map, a restaurant menu, a competition entry, or a reservation page for a special event are all possible uses for newspaper AR.

Newspapers around the world have added augmented reality in an attempt to enhance their product, although this has sometimes been done on a trial basis. The newspapers want to attract new audiences—especially young people—to increase their waning number of readers. Two papers in my area have used AR in recent years. All of the pictures in this article were taken by me as I explored augmented reality in the Metro and Burnaby NOW newspapers.

The Metro Newspaper

The Metro was a free weekday newspaper in the Greater Vancouver area that for a while included daily AR content. After the first edition of this article was written, the name of the newspaper was changed to Star Metro Vancouver, which distinguished it from the Metro newspapers published in different cities and countries. At least some of these newspapers have also used AR.

In my exploration of the newspaper's augmented reality features, I encountered YouTube videos, movie clips, videos that seemed to have been created by the newspaper, extra photos, photo slide shows, and an informative poster about training for a marathon. The newspaper also offered a competition code that was available through AR. The augmented reality was provided via Metro's own app for mobile devices.

Burnaby NOW

Burnaby NOW (part of the NOW newspaper group) is a free community newspaper that is published twice a week. At one time, each issue offered AR features similar to those of the Metro newspaper. The paper also enabled people to buy concert tickets through augmented reality. Burnaby NOW used the Layar app to provide its AR.

The Tokyo Shimbun is a Japanese newspaper that has used AR to change articles aimed at adults into easily understandable articles for children. Animated cartoon characters, color, pop-up headlines, and a simplified alphabet made the newspaper useful and entertaining for a child. In Australia, one newspaper group has used an AR app called News Alive to enable readers to explore a 3D version of buildings.

When the Metro published its first newspaper containing AR content, it likened the newspaper to the ones appearing in the Harry Potter movies. In these newspapers, moving scenes appeared on a page containing otherwise static content. This type of newspaper would be wonderful in real life, but it doesn't exist (yet). The appearance of movement on a page of today's newspapers is a trick that requires special equipment.

The reader must have a smartphone or tablet with a camera as well as an Internet connection in order to use the AR features of a newspaper. In addition, the augmented reality software must be able to link to the software controlling the camera of the mobile device. This should be no problem when using an iOS or Android device.

Augmented reality in a newspaper works via image recognition. With the aid of the camera app in the mobile device, the AR program identifies a photo and loads the digital content that is linked to that photo. As viewed through the camera, the linked content often appears as an overlay on top of the image that was scanned. If the digital content is a video, it does give the newspaper page a slightly Harry Potter-like appearance as it plays. Once the content is loaded, when the mobile device is moved away from the newspaper the content stays in view.

The augmented reality in my local Metro newspaper was marker based. The steps in using the AR from this newspaper on a mobile device such as my iPad were as follows. The newspaper app was available as a free download at the Apple Store.

1. Open the newspaper app.
2. Click the AR symbol on the front page of the app.
3. The app opens the camera application on the iPad in scan mode. Fill the screen with the photo to be scanned.
4. The photo is automatically scanned as a vertical green line moves over the photo. Markers (the green dots in two of the photos below) are temporarily laid down as the scan line moves.
5. Information from the markers is sent to a newspaper computer.
6. The computer compares the information from the scanned photo with the photo information stored in its database until it finds matching data.
7. Once a match is found, the computer performs the action that it's programmed to carry out when that photo has been identified (such as loading a particular video or slide show).

This process for obtaining AR content is fast when it works properly, as it often does. Sometimes the "Scan result not found" message appeared during my trials of the Metro AR, however. The scan line moved over the photo and markers appeared and then disappeared as normal, but the scan wasn't successful.

The augmented reality feature in the Burnaby NOW newspaper worked in the same general way as in the Metro newspaper. The Layar app used by Burnaby NOW examined outlines and edges in photos. It often worked very well but sometimes failed to scan successfully.

When a photo scan quickly triggers interesting content to load, the AR feature is very enjoyable. When multiple scans are needed to obtain success the process is less enticing. The photo scanning and content loading procedures aren't foolproof. It's frustrating when they don't work, whatever the reason for the problem. I occasionally encountered a photo that I couldn't scan correctly, even when I tried on multiple days and under multiple conditions.

Augmented reality in newspapers is potentially a very useful feature and could be a great enhancement to them in the future. I enjoy looking at digital content linked to newspaper articles. Based on my experience, however, the technology needs to be improved. I consider the following features to be very important in order for AR in newspapers to be effective.

• The augmented reality feature should be easy to use, reliable, and as foolproof as possible.
• The scanning and content loading process should be rapid. Instant or very nearly instant gratification is necessary in order for the technology to appeal to people.
• The digital content should offer added value to the print article.
• Ideally, the digital content should be obtainable only through the AR program and shouldn't be something that a person could find on the Internet on their own.
• If the content is available on the Internet and can be accessed via a web browser, it's very important that obtaining it through an AR program is a rapid and convenient process.
• Some people feel that the new content should contain movement or interactivity, since otherwise static material in the newspaper is simply being replaced with more static material. I don't mind seeing static digital content as long as it's rich in new information.

One question that needs to be answered is whether people will want to look at digital content linked to newspapers when they could get all their information from the Internet and dispense with a newspaper altogether. Compelling or unique content in both the paper and its AR content might help to attract readers.

Apparently the attraction of augmented reality—at least as it was implemented—wasn't strong enough in my local newspapers, since the feature was dropped. Improved technology and new abilities might encourage papers to try AR again. That won't help the Star Metro, though. The newspaper recently ceased publication. The publisher said that it was no longer financially advisable to create a free newspaper every weekday because the information in the paper was available on the company's website.

Other types of print media besides newspapers have used augmented reality, including the book version of Guinness World Records. Additional books and some magazines have used the technology as well.

Specialized AR apps are appearing. Microsoft Translator enables a traveler to point their mobile device at text written in another language and then read the text in their own language. The Google Translate app does the same thing.

Augmented reality apps have been developed to let people learn more about buildings, cars, museums displays, golf courses, mountains, stars, and other parts of the environment. Some of these apps work by detecting location instead of by reading markers, but all display digital content in addition to real-life content.

Eyeglasses that provide augmented reality seem to be increasing in popularity. The Pokémon Go game with added AR was very popular when it first appeared. There have certainly been some interesting developments, including the release of an AR development toolkit from Apple. Whether the new developments are helpful for newspapers remains to be seen.

The potential uses of augmented reality technology are exciting, but as is the case with many other technological developments, they aren't available to everyone. Many people don't have a smartphone or a tablet with mobile Internet access. They either can't afford them or don't want to get them. In some areas, the mobile devices or an Internet connection aren't available.

People involved with the creation and use of AR features are getting excited about the increasing number of people with smartphones and tablets and about the potential uses of augmented reality. It's predicted that new uses for AR will appear this year. The problem of supplying mobile devices for augmented reality to everyone who would like them or could benefit from them—such as children in schools—is a major one, however.

In the high school where I used to teach, there were many "haves" with their own mobile device but also a significant number of "have nots". The school discussed getting a class set of tablets but never bought them because it would have been an expensive purchase. The high cost is a shame because augmented reality could be very useful in education.

I'm looking forward to seeing new developments in augmented reality for newspapers, other publications, and the environment and am very interested in its potential. At the moment I'm not confident in the success of AR for newspapers, however.

In its present implementation, viewing the AR features of a newspaper requires a commitment that people may not be willing to make. A newspaper reader must open an app, tap on the screen (in at least some apps), hover their device over an image, get a successful scan, and wait for digital content to be loaded from a computer on the Internet.

This multistep process for obtaining digital content may be acceptable when all the steps work rapidly and seamlessly, but not when one or more of them causes a bottleneck. Viewing augmented reality needs to be a "hover and see" process in order to attract people.

If AR technology is improved, however—which it probably will be—its present applications could become exciting and new applications could appear. I hope the improvements come in time to help newspapers. I enjoy reading certain papers even when they have no augmented reality features and would be very unhappy if they disappeared. For me, they add value beyond the Internet.