Why Spotify Wants to Monitor Your Emotions

Streaming giant Spotify wants to play the music that fits your mood by reading your emotional state.

The company has patented technology that lets it monitor your voice and suggest tunes based on your "emotional state, gender, age, or accent." The patent granted last month lets Spotify "make observations" about a user's environment and emotions using speech recognition technology. Some experts say gathering information on users’ moods is a privacy risk. 

"Our identities are becoming more and more visual, and any personal or sensitive information that is stolen from a user can have devastating results for an individual," Mikaela Pisani, senior data scientist at software company Rootstrap, said in an email interview.

"For example, having certain control of a person's emotions is a very powerful tool, which in turn could prove easy to influence users or to have a certain type of control over markets and politics."

Getting to Know You

According to the patent application, Spotify’s software uses speech recognition to gather information about age and gender. It would monitor the "intonation, stress, [and] rhythm" in the user's voice to identify if someone was "happy, angry, sad, or neutral."

Not everyone agrees that emotion-tracking software is a privacy risk. "The ethical and legal ramifications seem overblown in proportion when one considers how companies have historically been using customers' data and personal information without much ado," Scott Hasting, co-founder of BetWorthy, a company that develops sports betting software and deals with emotion detection, said in an email interview.

"These new techniques may actually bring about more benefits, mainly due to the opportunity to build better and more trusting relationships with the end-users," Hasting said. 

It’s not just the music industry that wants to know how you are feeling. Some companies are using software to gauge employees’ moods, and it’s raising ethical questions. "The goal should be to address issues and to maintain a positive culture and engaged employee base," Mike Hicks, chief marketing officer of intelligent digital workplace company Beezy, said in an email interview.

"But many employees won’t see it that way. Will I be penalized if my mood is consistently low? Will this information go in my employee file? Will teams and individuals with low moods be let go? Am I sharing too much of my personal life with my employer if I accurately measure my mood multiple times per day?"

All It Takes Is a Microphone

Potentially, any application that has access to a microphone and uses speech recognition can determine characteristics of a user, including gender, age, and race, Pisani said. Software even can identify personality traits.

"For example, it can pick up on if a user is passionate about a certain song or not, or if it is angry about a certain particular topic," she added. "Certain applications can even examine users’ breath, such as the ones that track sleep."

Your feelings also can also be revealed through apps that use a camera, Pisani said. 

For example, Realeyes' software can monitor emotions from users' faces. There’s also the app Breathe2Relax, which can process people's mood information and potentially help to manage depression and anxiety. Affectiva offers software that can monitor drivers to determine how tired and distracted they are, preventing accidents.

"You could argue that AI now knows people better than they know themselves, and this comes with its own advantages and disadvantages," Pisani said. 

"This is beneficial for users who prefer to have their decisions made for them." But, she pointed out, "most apps are not made for maintaining a users’ health."