Do Augmented Reality Patents Hint at Future Apple Products?
Apple’s growing arsenal of patents related to augmented reality provide an interesting look at a direction that the company’s lineup of consumer electronics could take in the future. One of Apple’s latest patents, first spotted by Apple Insider, is just one of an ongoing series detailing the company’s ideas for possible applications of augmented reality technology.
Apple’s patent number 8,890,771, for a “Transparent electronic device,” describes a device with one or more screens that would allow a device to overlay images over real-world objects, using a “visible window” on an otherwise opaque screen. “Additionally,” the patent reads, “the display may include active and passive display screens that may be utilized based on images to be displayed.”
The patent explains that the device could possibly be a tablet or a mobile phone. The device would enable the user to see a physical object on one side of the electronic device through the transparent screen on the other, and the display would be configured “to display image data such that the image data at least partially overlays the physical object.”
The patent says that the technology is a response to “a need for displays that may provide increased functionality to a user.” Both passive and active displays could be used, and in one embodiment, the display could include a transparent window that the user could move, as needed, via multitouch input. The patent also describes a “black mask” that the device could use to hide the internal electronic components from view.
The patent explains how the user might interact with the device, and outlined a few examples of how augmented reality could combine with a transparent display to provide useful information to the user:
By utilizing a touch screen in conjunction with the display screens 48 and 50, a user may be able to interface with the electronic device 10 by touching portions of the display 14, which may correspondingly move the location of the interface indicator 52 generated on display screen 48. … Other overlays in addition to or instead of the interface indicator 52 may be generated on the display screen 48. For example, overlays that include icons, text, graphics, or other generated images may be generated on the display screen 48.
These overlays whether in handheld or other electronic devices 10, may provide an “augmented reality” interface in which the overlays virtually interact with real-world objects. For example, the overlays may be transmitted onto a display screen that overlays a museum exhibit, such as a painting. The overlay may include information relating to the painting that may be useful or interesting to viewers of the exhibit. Additionally, overlays may be utilized on displays in front of, for example, landmarks, historic sites, or other scenic locations. The overlays may again provide information relating to real-world objects as they are being viewed by a user.
This is the latest patent that provides a view into Apple’s thinking on how augmented reality may play out in future consumer electronics. As Apple Insider reported in September, Apple filed two patent applications describing “federated mobile device positioning” and “registration between actual mobile device position and environmental model,” which both described an augmented reality system that would use the iPhone’s camera, sensors, and communications functionality to layer rich location data over a real-time world view.
The system uses GPS, Wi-Fi signal strength, sensor data, and other information from the phone to determine the user’s location. An app then downloads a three-dimensional model of the area, with data on nearby buildings and points of interest. The virtual frame of the model is overlaid a live video feed from the iPhone’s camera, and users would be able to manipulate the model onscreen with gestures or with spoken commands.
Once the image is calibrated and the wireframe is locked in, the system streams information to the user via onscreen overlays. At that point, users would be able to read the names of nearby buildings and roads, find information on nearby businesses, and possibly “peel back” walls to view the interiors of a selection of buildings.
As Patently Apple reported in September, Apple was granted patent number 8,832,557 for a personal display system and the functionality for adjusting the media displayed on it, or “Adjusting media display in a personal display system based on perspective.” The technology could be implemented as a head-mounted display, a television, a projection screen, an on-screen display of an electronic device, or as a personal viewer or viewfinder on devices like digital cameras.
CNET last year spotted another augmented reality patent, number 8,400,548, for “Synchronized, interactive augmented reality displays for multifunction devices.” The system that the patent describes captures a live video feed of an object to display on the device and overlays information about that object on the device.
An example cited in that patent explained that the user could point the device’s camera at a computer’s circuit board, and the feed would display an image of the board overlaid with annotations on its parts. The display would also enable users to create their own annotations by writing or drawing on the device, as well as share their annotations with others.
Could Apple’s growing array of patents mean that an augmented reality-capable iPhone is in the future? It’s impossible to tell, and worth noting that the technology detailed in many of Apple’s patents never makes it to a consumer product. But it seems that Apple is serious about investigating what it would take to bring augmented reality to iOS devices, and Apple and other Silicon Valley giants seem to be laying the foundation for a new generation of devices that will further blur the line between the virtual and physical worlds we all inhabit.