Amazon, Apple, Facebook, Google, and Microsoft: They’re some of the biggest names in the tech world, and have a gravitational pull not only over headlines, but over the gadgets and apps you use everyday. Nowhere is that better illustrated than in the competitive world of virtual assistants, which handle tasks in response to your voice commands, your jokes and philosophical questions, or even typed queries. While Siri, Google Now, and Cortana all have their limitations and misunderstandings, the idea is to build an artificially-intelligent assistant that will be able to help you more often than not. So which virtual assistant is most capable? Which makes it easiest to imagine that we’ll eventually have software that can approximate the conversational skills of Samantha, the intelligent OS from Spike Jonze’s movie Her?
The New York Times’s Farhad Manjoo recently christened the competitors in this battle of tech giants (and virtual assistants) the “Frightful Five,” noting that they’re “getting larger, more entrenched in their own sectors, more powerful in new sectors and better insulated against surprising competition from upstarts.” These companies are in a unique position to pit their virtual assistants against one another because each one has built several core technologies, which have become central to everything we do on computers and smartphones.
The result is that these companies own many of the world’s most valuable platforms, platforms that Manjoo notes are “inescapable” — you can choose not to buy an iPhone to avoid Apple’s platform, for instance, but then you’re probably going to end up with an Android phone and Google’s platform. Together, Manjoo explains, “they form a gilded mesh blanketing the entire economy.” These platforms are what enable each member of the Frightful Five to build a virtual assistant and immediately have an audience for it, whether or not the virtual assistant in question is really able to stand up to the competition.
Brian X. Chen graded the performance of the competitors — Amazon’s Alexa, Apple’s Siri, Facebook’s M, Google’s Google Now, and Microsoft’s Cortana — on tasks in categories that most consumers enjoy, like music, productivity, travel and commuting, dining, entertainment, and sports. But he found that “none of the voice assistants earned a report card that would make a strict parent proud.” In terms of grade-point averages out of 4.0, Google Now scored 3.1. Siri earned a 2.9. Cortana achieved a 2.3. And Alexa scored 1.7.
Google was the best at travel and commute-related tasks. It responded with useful information to queries about traffic en route to a location, and showed how long it would take to drive to the destination. Asking for directions yielded voice directions and a map, and asking for plane tickets for the following week yielded a starting price and information on the duration of the shortest flight.
Siri was the strongest at productivity tasks, including calendar appointments and email. The assistant was able to schedule a meeting, check what was on Chen’s calendar, send an email, and read the most recent email aloud. Other assistants could complete only some of those tasks. While Siri performed well at music-related tasks, she was bested by Alexa. Google Now and Siri performed equally well at food-related tasks.
Cortana, in Chen’s experience, was “mediocre across the board.” The assistant faltered when asked about flight information, couldn’t offer traffic estimates, and offered directions to the wrong destination. But when asked about who would play in the upcoming Super Bowl, Cortana was the only assistant aside from Google Now to deliver the information that the Carolina Panthers would face the Denver Broncos (instead of giving information about when and where the game will be played, or loading scores for Sunday football games).
Alexa performed best at music-related tasks. Unlike Siri, Alexa could play a specific music station on Pandora upon request. But she was less helpful at other common tasks, and couldn’t offer traffic estimates for any location other than the one fixed location in the app (like your office).
Facebook’s M was left out of the grading system because the company denied Chen access to its virtual assistant, which it’s testing with only a small number of users. M is controlled in part by artificial intelligence and in part by human operators, and you talk to the assistant by sending messages via Facebook Messenger. Chen suspected that a person handled most of the questions that he posed, concluding that M is more capable than all the other virtual assistants largely because “humans are on the other end of the puppet strings handling tasks that artificial intelligence cannot.”
While it might be difficult to conclude which virtual assistant is smartest, it’s a little more straightforward to address another question that many users have asked about the assistants offered by the Frightful Five: which one is most considerate of your privacy? As J.D. Biersdorfer reports for The New York Times, personal assistants need access to information about you to complete the chores you task them with, whether that’s your contacts list so they can send messages for you, your location for finding businesses near you, or your calendar so that they can add events.
When you use voice commands with a personal assistant, your audio data can also be sent to the company’s servers for processing, and for improving the system’s voice recognition abilities. Companies can also collect your browsing activity and search history, which they say helps to provide more relevant results, but in reality may also be used to help send targeted advertisements.
The upshot is that if you want the functionality of a virtual assistant, you’re going to have to loosen your grip on your privacy. But policies differ company to company. While Google is notorious for using your personal data to target ads to you, and may associate your device identifiers or phone number with your Google account, you might find it reassuring that while Siri sends some information to Apple’s servers, that information isn’t associated with your Apple ID, and your searches and locations can’t be traced to you. And Microsoft enables you to use Cortana in a limited way in order to retain control over the kind of information that the assistant collects.
Ultimately, using a capable virtual assistant and expecting useful responses to increasingly complex queries is going to require affording the software access to at least some information about you, and granting it the permission to share data with the servers back at HQ. While most people are more likely to be comfortable with policies like Apple’s than those put in place by Google, the real question is whether virtual assistants’ abilities will advance quickly enough to make you feel alright about overlooking the eavesdropping.