What a Virtual Assistant is and How it Works

How smart speakers and assistants are transforming our lives

Man looking at tablet

A virtual assistant is an application that can understand voice commands and complete tasks for a user. Virtual assistants are available on most smartphones and tablets, traditional computers, and, now, even standalone devices like the Amazon Echo and Google Home.

They combine specialized computer chips, microphones, and software that listens for specific spoken commands from you and usually answers back with a voice that you select.

While virtual assistants can also refer to people who perform administrative work for others, such as setting up appointments and submitting invoices, this article is about the smart assistants that live in our smartphones and other smart devices.

The Basics of Virtual Assistants

There are five primary virtual assistants on the market (others exist but aren't as popular):

Virtual assistants like these can do everything from answer questions, tell jokes, play music, and control items in your home such as lights, thermostat, door locks and smart home devices. They can respond to all sorts of voice commands, send text messages, make phone calls, set up reminders; anything you do on your phone, you can probably ask your virtual assistant to do for you.

Even better, virtual assistants can learn over time and get to know your habits and preferences, so they're always getting smarter. Using artificial intelligence (AI), virtual assistants can understand natural language, recognize faces, identify objects, and communicate with other smart devices and software

The power of digital assistants will only grow, and it's inevitable that you'll use one of these assistants sooner or later (if you haven't already). Amazon Echo and Google Home are the main choices in smart speakers, though we expect to see models from other brands down the road.

How to Use a Virtual Assistant

In most cases, you'll need to "wake up" your virtual assistant by saying their name (Hey Siri, OK Google, Alexa). Most virtual assistants are smart enough to understand natural language, but you have to be specific. For instance, if you connect Amazon Echo with the Uber app, Alexa can request a ride, but you have to phrase the command correctly. You have to say "Alexa, ask Uber to request a ride."

Typically you'll need to speak to your virtual assistant because it's listening for voice commands. Some assistants, however, can reply to typed commands. For example, iPhones running iOS 11 or later can type questions or commands to Siri rather than speaking them. Also, Siri can respond by text rather than speech if you prefer. Likewise Google Assistant can respond to typed commands by voice (choice of two) or by text.

On smartphones, you can use a virtual assistant to adjust settings or complete tasks such as sending a text, making a phone call, or playing a song. Using a smart speaker, you can control other smart devices in your home such as the thermostat, lights, or security system.

How Virtual Assistants Work

Virtual assistants are what's called passive listening devices that respond once they recognize a command or greeting (such as "Hey Siri"). This means the device is always hearing what's happening around it, which could raise some privacy concerns, as have been highlighted by smart devices serving as witnesses to crimes.

The virtual assistant must be connected to the Internet so it can conduct web searches and find answers or communicate with other smart devices. However, since they are passive listening devices, they usually need a wake work to activate. That said, it's not unheard of that the device could start recording when a wake word hasn't been used. In fact, it was a case about an Echo that was witness to a murder that brought this fact to the public's attention.

When you communicate with a virtual assistant by voice, you can trigger the assistant and ask your question without pausing. For example: "Hey Siri, what was the score of the Eagle's game?" If the virtual assistant doesn't understand your command or can't find an answer, it will let you know, and you can try again by rephrasing your question or speaking louder or slower. In some cases, there may be some back and forth necessary, like if you ask for an Uber, you may have to provide additional information about your current location or destination.

Smartphone-based virtual assistants like Siri and Google Assistant can also be activated by holding down the home button on your device. Then you can type in your question or request, and Siri and Google will respond by text. Smart speakers, such as the Amazon Echo can only respond to voice commands.

The Popular Virtual Assistants

Alexa is Amazon's virtual assistant and it is available on the Amazon Echo line of smart speakers as well as third-party speakers from brands including Sonos and Ultimate Ears. You can ask the Echo questions like "who is hosting SNL this week," ask it to play a song or make a phone call, and control your smart home devices as you can with most virtual assistants. It also has a feature called "multi-room music," the lets you play the same music from each of your Echo speakers, much like you can do with Sonos speaker systems. You can also configure the Amazon Echo with third-party apps, so you can use it to call an Uber, pull up a recipe, or lead you through a workout.

Samsung's take on virtual assistants is Bixby, which is compatible with Samsung smartphones running Android 7.9 Nougat or higher. Like Alexa, Bixby responds to voice commands. It can also give you reminders about upcoming events or tasks. You can also use Bixby along with your camera to shop, get a translation, read QR codes, and identify a location. For example, take a picture of a building to get information about it, snap a photo of a product you're interested in buying, or take a photo of text that you'd like translated to English or Korean. (Samsung's headquarters are in South Korea.) Bixby can control most of your device settings and can mirror content from your phone to most Samsung Smart TVs.

Cortana is Microsoft’s virtual digital assistant that's comes installed with Windows 10 computers. It's also available as a download for Android and Apple mobile devices. Microsoft has also partnered with Harman Kardon to release a smart speaker. Cortana uses the Bing search engine to answer simple queries and can set reminders and answer voice commands. You can set time-based and location-based reminders, and even create a photo reminder if you need to pick something specific up at the store. To get Cortana on your Android or Apple device, you'll need to create or log into a Microsoft account.

Google Assistant is built into Google Pixel smartphones, the Google Home smart speaker, and some third-party speakers from brands including JBL. You can also interact with the Google Assistant on your smartwatch, laptop, and TV as well as in the Google Allo messaging app. (Allo is available for Android and iOS.) While you can use specific voice commands, it also responds to a more conversational tone and follow-up questions. Google Assistant interacts with a multitude of apps and smart home devices.

Finally, Siri, perhaps the most well-known virtual assistant is Apple's brainchild. This virtual assistant works on iPhone, iPad, Mac, Apple Watch, Apple TV, and HomePod, the company's smart speaker. The default voice is female, but you can change it to male, and change the language to Spanish, Chinese, French, and a few others. You can also teach it how to pronounce names correctly. When dictating, you can speak out the punctuation and tap to edit if Siri gets the message wrong. For commands, you can use natural language.