RealNetworks Brings Live Facial Recognition to YouTube and Netflix

The venerable RealNetworks is ready to ready to rejoin the consumer race

The last time I last saw RealNetworks Founder and CEO Rob Glaser, almost two decades ago, we were sitting in a small conference room at CES with Dana Brunetti and Kevin Spacey (yes, that Kevin Spacey) talking about early streaming technology and user-generated video content. That was back when RealNetworks was a major player in the early video download and streaming arena. When the pugnacious Glaser would duke it out with tech behemoths like Microsoft.

Simon Baker and Demi Moore
 RealVideo

The man on the other side of our video call was not the Glaser from the past and RealNetworks is barely a blip in the consumer psyche. RealNetworks primarily focuses on vertical solutions like SAFR, its live video facial recognition solution, and Napster, its gaming and some consumer media. Glaser, like me, is grayer and maybe more introspective and started the call by checking in on how I was holding up during a pandemic and global unrest.

I enjoyed catching up, but we had other matters to discuss. Glaser and RealNetworks and getting back into the consumer game in a significant way, delivering what may be the first real-time facial recognition technology for YouTube, Netflix, and personal video.

Rob Glaser
RealNetworks CEO Rob Glaser.  RealNetworks

I haven’t thought much about RealNetworks in the last decade and, to be honest, was unaware that it’s made the shift to delivering cloud-based facial recognition technology to security companies and law enforcement (outside the U.S.). RealNetworks hasn’t entirely left the consumer arena. The company now boasts 1 billion total downloads of its RealPlayer software and still enjoys millions of fresh downloads each year. Many people apparently still use RealPlayer to manage personal video collections and to, though it seems odd to me, download YouTube video content.

There are, however, few online discussions about how to use or get more out of RealPlayer. If it has a devoted userbase, they are incredibly quiet. That may change, though, with RealPlayer 2020 and the companion online video facial recognition service: StarSearch.

The journey here started a year ago when RealNetworks set out to figure out how it could combine its SAFER computer vision technology with commercial video to “create a nice, breakthrough product,” said Glaser.

Hello PC

The Pandemic, Glaser told me, has put the PC back in the center of consumer video. It is, he noted, where we host our Zoom Meetings and stream Netflix with friends through Netflix Party. “People want to socialize, and the PC is able to do with browser extensions what you can’t do on TV,” he added.

As a result, RealNetworks' decision to bring its latest innovations to the PC and Web browser, as opposed to apps on streaming boxes and smart TVs, makes some sense.

StarSearch

I don’t know if Glaser had to clear it with the estate of Ed McMahon, but the name StarSearch makes perfect sense when you consider that the browsers extension, which currently only works in Chrome (Microsoft Edge support is coming), literally finds celebrity faces in videos on YouTube and Netflix.

YouTube Video
StarSearch found the 20 celebrities who participated in this Lord of the Rings reunion special, and all the times the appear in the video.  YouTube

The free browser extension, which is available for download from RealNetworks' site starting today, is the picture of simplicity. As soon as you install it, StarSearch is ready to identity actors, sports stars, YouTube celebrities, musicians, and politicians.

I downloaded a StarSearch preview and loaded Josh Gad’s recent Lord of the Rings cast reunion YouTube video. Near the upper left corner of the playback window, I saw a small blue circle with the number 20 in it. This meant StarSearch and its underlying SAFER facial recognition technology had identified 20 celebrities.

Dakota Johnson
Clicking on a thumbnail brings up an info card and a link to more videos featuring the actress.  YouTube

I clicked on the circle and a vertical list of all the identified stars appeared. Each celebrity thumbnail including a link to more info (culled, for now, from Wikipedia) and the number of times they appear in the video. When I clicked the number, I could see a timeline with thumbnails hovering over the exact spots in the video where the celebrity appears.

I could also drill a level deeper and find a list of other videos featuring the same actor.

It was fast and impressive.

So far, RealNetworks has indexed just 30,000 YouTube videos (the platform has billion of videos), focusing primarily on trending and top videos. In those videos, they’ve identified roughly 15,000 faces, but hope to eventually build a database of hundreds of thousands of them.

I didn’t have as much luck with Netflix. In fact, while the blue circle appeared consistently on all video, StarSearch failed to identify anyone in more than half a dozen streaming shows. I did see a demo where the service worked on Netflix's Ozark.

My guess is RealNetworks focused its initial indexing on YouTube and is playing catchup with Netflix. Glaser and his team told me they plan to add support for Hulu, Disney+ and other streaming services. The extension does not yet work on videos within Twitter.

Chris Pratt
This compilation video got 27 facial identification hits.  YouTube

If StarSearch’s on-the-fly celeb identification seems familiar, it’s because you’ve seen something quite similar on Amazon Fire TVs. Those streaming devices include Amazon X-Ray, which identifies exactly who is in each scene. It does not offer a broader, list every actor in this entire video and let me jump to each of these scenes, feature, though.

RealPlayer 2020

RealNetworks is also taking that same SAFR facial recognition technology and folding it, for the first time, into its RealPlayer video management utility. It’s under the People section and, once you select which folders to watch, the system ingests your video and identifies and organizes faces throughout it.

While the idea is akin to what you’d find in people searches in Apple Photos and Google Photos, the interface is more purely RealNetworks and very much a PC approach. In other words, it’s a bit clunky, but workable.

RealPlayer 2020
RealPlayer 2020's interface is what I would call functional.  RealVideo

When I installed the RealPlayer 2020, it guided me through selecting all the folders I wanted it to watch for existing and new video content. Then it set about indexing it. My problem is that I have very little personal video on my PC (this is 2020, after all). Instead, the lion’s share of my recent video is what I’ve shot with my various iPhones. It all lives either on my phone or in iCloud, which means RealPlayer can’t touch it.

Eventually I connected an old backup drive to my laptop to see what RealPlayer could find. It’s still indexing the hundreds of pieces of content.

RealPlayer 2020 is free, but there are upgrade paths that add a variety of features including HD downloads and saving music from videos.

Nuts and Bolts

SAFR, the technology at the heart of RealNetworks’ People search is a cloud-based facial recognition system. For YouTube, the system is indexing entire videos, analyzing frames, and either indexing new faces or matching faces against the existing index. In the case of Netflix, Digital Right Management (DRM) prevents RealNetworks from indexing the videos, so it attempts to identify faces on the fly. Perhaps this is why my Netflix StarSearch sessions was so unsuccessful.

'This is the most horizontal thing we’ve done in streaming in 10 years or maybe longer.'

For your personal library, the system gets to work, with your permission and, for privacy purposes, handles all facial recognition tasks locally. Glaser explained that the system is “privacy sensitive,” adding that what they do is “100% legal in every state in the country except Illinois.” In fact the company had to build a geofence around their app so the People feature would not work in The Prairie State.

While RealPlayer's system analyzes your video, it does not copy it. After scanning the video, it creates a small metadata file with all video's relevant indexing info relating to the people and where they appear in the video.

It can take, on a Core i3 system, an hour to import, analyze, and index an hour of video. Faster systems can cut the process by half or more. But if you have a big video library, you’ll want to be very patient.

RealPlayer People Search
It took more than 12 hours to get this far through my video library. The sysetm identified some people, but could not always identify duplicates.  RealPlayer

Once the system does find faces in your video, you’ll see thumbnails for each person and, if you double click, you’ll see thumbnails for each video in which they appear. In the video, you can, as in StarSearch, see exactly where in each video people appear. If you wanted, for example, to find among hours of video that brief clip of grandma, you could instantly jump to her star-making appearance.

People in RealPlayer 2020 does let you label the faces you find, but it’s not like Google Photo, which helps you figure out if two thumbnails are of the same person, but with different haircuts or at different ages.

RealPlayer
With RealPlayer's people search, you can find the face you're looking for in hours of video.  RealVideo

However, once you have labeled people, you can perform more complex searches to find, for instance, videos in which both you and grandma appeared.

There are other changes in the RealPlayer, like a new UI and color scheme, but having not seen RealPlayer in 15 years, it’s all new to me. I would say it’s a mix of old school PC grittiness and some nice 21st century polish. The real star of the show, though, is the People tool, which, for those with hard drives full of old videos (and maybe libraries of personal DVDS), this could come in very handy.

Bottom Line

RealPlayer 2020 and, especially, StarSearch offer some exciting possibilities for finding a face among the weeds of a billion hours of online video and hundreds of hours of personal and downloaded video. The browser extension is smart way to quickly reintroduce people to the RealNetworks brand, without overwhelming them with a wonky utility.

Glaser reminded me that Real hasn’t been gone, but it is, with these updates, sort of reemerging. “This is the most horizontal thing we’ve done in streaming in 10 years or maybe longer.”

I don’t know if people are so desperate to index their own video or if they even have enough of it in the right place to gather any real value off RealPlayer 2020, but the SAFR-powered StarSearch capability is probably at least enough to get RealNetworks back into the consumer conversation. Whether it can stay there and regrow the consumer brand to past glory is another matter.