How Fluent Voice Recognition Stays Fast, Accurate, and Private

Fluent.ai is a virtual voice recognition engine that doesn’t send your commands off to the internet, acts almost instantly, can work in any language, and is so small that it can be built into even cheap, low-power devices like a fitness watch, for example.

Unlike Siri and Alexa, Fluent is a self-contained assistant that understands you instantly and learns from you so it gets better the more you use it. It doesn’t have the depth of regular virtual assistants, but it’s not meant to. Instead, it is faster, more accurate, and more private than Apple, Amazon, and Google’s efforts.

"Speech to text uses natural language processing, and then derives intent," Fluent CEO Probal Lala told Lifewire during a Zoom interview. "It requires a lot of data, and a lot of processing power. Fluent goes directly from speech to intent, taking your voice and converting it directly into an action."

What is Fluent?

Fluent is voice-control software. It works by listening to your command and stripping out all the words it doesn’t need, leaving just essential nouns and verbs. "Turn out the lights," becomes just out and lights. The essential elements are stripped from a messy human sentence and turned into steps. It’s almost like programming a computer, turning a complex idea into the simplest set of instructions.

What Can It Be Used For?

Fluent can be used for anything. The key is that it is trained for a specific situation. With a smart watch, for example, it might be trained in commands for fitness, or for home automation, calendar and timer creation, and so on. Limiting the database makes everything more focused and keeps it fast. 

"The truth of the matter is that with wearables, you're not looking to have a conversation," said Lala, "I want maybe my favorite exercise routine to begin, and want it very quickly."

Speed and accuracy are the whole point. Switching lights on and off takes milliseconds, instead of having to wait for Siri to send your voice to the cloud, wait for it to be processed, then—seconds later—having the lights go off.

This narrow training also keeps the size of the app down. Last year, Google provided a downloadable, offline version of its assistant. It was, says Lala, 85 Megabytes, only worked in English, and it took six months to train it.

"We've got one model working with 13,000 commands and it operates in 500 kilobytes," he said. 

Privacy

The other advantage of an offline voice recognition engine is privacy. In a kid’s smart watch, for example, "you don’t want a child’s voice going to the cloud," says Lala. In fact, Fluent can work inside devices that never connect to the internet. This is not only good for privacy, but also security. You could use an offline voice recognition engine inside research labs, military installations, and other places that ban cell phones and cameras. 

Limits

Of course, there are some downsides to this model. One is that commands cannot be added later. Once the initial training is complete, that’s it. Nor can your Fluent-based assistant look things up on the internet for you—sport scores, the actor you recognize in that movie, but just can’t place, and so on.

Instead, the system is smart enough to recognize when it can’t help and it will hand the request off to something that can. If you ask your watch for the weather forecast, Fluent will realize that it doesn’t understand. "Then it will call a service that's preprogrammed in the watch, whether it be Amazon's Alexa or Google, and then make a call to the cloud," passing on your raw voice command to get the answer. 

This hybrid approach keeps the speed of a local, offline assistant, with the power of Alexa or Google Assistant as a backup. 

Can You Buy Any Fluent Devices?

Not yet. The company is licensing its tech, and doing the training, for other companies. Thanks to COVID, two major launches have been pushed back to next year. But you can expect to see it showing up in watches and other fitness devices, speakers, home automation hubs, and so on.

It would be great if this was also incorporated directly into smartphones like iPhone, making day-to-day tasks faster, and only resorting to Siri when necessary. That would be the real killer app.

Update: October 22, 10:12am. Made changes to refer to Fluent as a voice recognition engine rather than a voice assistant.