How AI Changes Our Perspective on Fine Art

Famed artist Wassily Kandinsky could match each musical note with an exact color, due to his rare neurological condition called synesthesia. Now, you can experience how Kandinsky saw the world thanks to a new online project. 

The Google project called Play a Kandinsky lets you hear what Kandinsky might have heard as he looked at color. The interactive tool enables you to experience his abstract 1925 masterpiece Yellow Red Blue through sound by clicking around the artwork to listen to a seven-movement composition that travels through colors and moods as Kandinsky described them.

"Google has taught an artificial neural network the correspondence of colors and shapes to sounds and emotions," Sergey Burukin, head of decision intelligence at web development company Greenice, said in an email interview.

"They’ve trained the system on Kandinsky’s music collection and got it to generate sounds the artist may have experienced from his paintings."

Artificial intelligence (AI) can deliver highly unexpected but startlingly accurate predictions, Erik von Stackelberg, chief design officer of the software firm Myplanet, said in an email interview.

"For medicine, it could mean accelerated diagnoses," he added. "But in art, it means the potential for what we as humans might perceive as variety, spontaneity, or previously unconsidered perspectives."

Create Your Own Masterpiece

The Play a Kandinsky web app lets you create your own artwork. The site asks you to select two emotions to hear your mood as inspired by Kandinsky. You are then presented with a Kandinsky picture that you can click on to listen to sounds. You also get interactive content that allows you to explore the history of Kandinsky’s work. 

"Music played an essential role in Kandinsky’s work," Renée B. Miller wrote in a blog post for the Denver Art Museum. Viennese composer Arnold Schönberg was one influence, she said.

"Schönberg abandoned tonal and harmonic conventions in his compositions the same way that Kandinsky rejected the figure or recognizable object in favor of shapes, lines, and discordant colors in his work," Miller wrote.

Art exhibits like the Kandinsky project are increasingly going online as the coronavirus pandemic limits attendance at museums and galleries. For example, the San Diego Museum of Art recently launched SDMA 360: A Virtual Gallery Experience, where visitors can explore galleries, zoom in to see art details, and read full label text in both English and Spanish. 

Go Virtual For the Best Art Viewing

The best way to view most online art collections is with virtual reality, Bryce Mathew Watts, an anthropologist who works with cultural institutions to digitize their work, said in an email interview.

"With this tool, guests can look at an object in the gallery, be transported to the studio where it was made or the site where the different elements were found," he said.

"By utilizing technology not just to copy and paste what is seen in-person, but to create a richer experience, this reaches a wider audience and provides something that goes beyond the physical space."

Other types of art venues also are moving online during the pandemic. Visual AIDS, an organization that uses art to illuminate the AIDS crisis, built a unique website that replicated the look of an art gallery with the artworks on the wall for its annual Postcards from the Edge benefit event. 

"We decided to create a simple website rather than a 3D virtual experience, to allow an easier viewing experience since there were over 1,000 small artworks to scroll through," Esther McGowan, the executive director of Visual AIDS, said in an email interview.

"Our web designers created a system that allowed viewers to click on works on the virtual 'wall' and have them enlarge, and then the medium also appeared so that buyers could get a better sense of how the work was made prior to deciding to buy it."