Multi-Dimensional Audio - Rethinking Surround Sound

Freeing Surround Sound from Channel and Speaker Limitations

Dateline: 06/08/2012
Updated: 5/23/2014

Over the past year or so, I have reported on efforts by SRS Labs (since purchased by DTS) to establish a new way of approaching the surround sound experience by spearheading the formation of the 3D Audio Alliance. As a result of these efforts, a new proposed open-source audio ecosystem has been developed, dubbed Multi-Dimensional Audio. The first platform specification 1.0 is now available for implementation for the entire audio production/reproduction process, from Content Creation and Content Distribution, to Content Consumption.

Audio in 3D Dimensional Space

The main breakthrough presented by the MDA standard is a re-conceptualization of audio as sound objects located in three dimensional space, rather than sounds anchored to a specific channel or speaker configuration, as it is in the traditional stereo and surround sound formats.

In other words, when a sound engineer is presented with the task of mixing the elements of a movie soundtrack or music performance, instead of having to place each element with deference to what channels and speakers are typically needed to reproduce the sound, or what format needs to be used for eventual playback, whether it be MDA, any of the Dolby Digital and DTS formats, SRS Circle Cinema 3D, etc... all the sound engineer needs to be concerned with is where sounds need to be located in three dimensional space. In the MDA system, each object (or group of objects) can be assigned its (their) own identity, allowing them to be manipulated independently in the mixing process, regardless of the number of channels or speakers associated with the playback format.

Channels and Speakers

With MDA, the channel structure used in recording and mixing is now just a foundation from which to build a more natural audio experience. Traditional PCM channel structure is still used to record, mix, and deliver the sound, but the sound engineer now adds metadata that carries information where each designated sound object is located in three dimensional space, beyond what is provided by the PCM foundation.

The combination of PCM and metadata information is referred to in the MDA structure as PCM+.

On the listening end, the user simply accesses the content, and the playback device (referred to as a renderer) reads the PCM+ information and places the encoded sound objects in three dimensional space in accordance with their location assignment and the capabilities of the playback device(s). No matter how many channels or speakers are used on the playback end, each of the audio objects within the MDA mix will be placed in the proper three dimensional location in relationship to each other and the listening position.

As a result, the number of speakers you have available to you work in a similar way as pixels on a video display (such as a TV or video projector). The more channels or speakers available in the playback system, the more accurately detailed the three dimensional spatial representation of the sounds will be. However, no matter how few (2 or 5) or many (7, 9, 11, 14, 22, 64, etc...) speakers you have, you will experience a representation of what the sound mixer intended, it is just preciseness and level of immersion provided is limited or augmented by the number of speakers and any additional audio processing in the associated equipment available to reproduce it.

In the MDA system, you don't need to mix for a specific number of channels or speakers.

Creative and Listening Freedom

Another element to the MDA system is not only does the original sound engineer have the ability to mix the audio soundtrack of a specific piece of content, but the end listener can also be given that capability. Using an interactive application interface (currently dubbed MDA Director), the listener can reposition or accentuate any of the isolated object elements in a soundtrack that have been designated as unlocked objects. This goes way beyond just changing sound levels of what comes out of each speaker - but actually goes into changing either sound levels or positioning of each object within the larger group of sounds that the speaker is reproducing. Think of it as volume or panning control for each designated unlocked sound object, not just for an entire channel.

Interactive Listening Scenario

One possible scenario to illustrate the interactive capabilities of MDA is with a football game broadcast. To set up MDA for a live broadcast the sound engineer would first assign which objects to be isolated, such as both an English and Spanish sportscaster, fans on each end of the stadium and in the middle, the cheerleaders, the band, and the players and referees.

With all these elements isolated, the fun begins (and I not talking just about the game). During the course of viewing the game, you may want to choose either the English or Spanish sportscaster, then you might want move the sportscaster's voice over to another part of the room, or you want to hear what the players are saying on the field more clearly, or perhaps you want to forgo the entire sportscaster voice over and just hear the crowd and the cheerleaders, and while your at it you want to put those cheerleaders off to the left, or the right, or behind you, or you can place yourself in the audience or outside the audience. Or, how about placing your listening position on the 50 yard line, or in one of the end zones? Of course, you also have the option of resetting the soundtrack back to what you had been originally presented with (but choosing either the English or Spanish sportscaster).

As you can imagine, such an experience might be able to be provided on a Blu-ray Disc, or other delivery format, where even more precision could be applied to the designation of sound objects.

MDA and Current Audio Technologies

Now, you are probably wondering: "Oh no, not another surround sound format to worry about! Ny home theater receiver already decodes Dolby, DTS, and even has THX post processing - so does that mean that I now have to buy another receiver to be able to play MDA-encoded content?"

The answer is Yes and No. Of course, if you want to get the entirety of the MDA listening or interactive experience, you need to have a home theater receiver, or similar component that can read the encoded spatial location metadata and distribute it in your listening environment accordingly.

However, you are not totally out of luck as MDA is an open source specification. This means that any sound engineer tasked with mixing the audio content for distribution can easily (at the touch of a button), translate an MDA-created mix to the Dolby, DTS, or any other surround sound format that is currently currently in use, just as standard Dolby Digital and/or DTS soundtracks are also provided on current Blu-ray disc releases for those that do not have Dolby TrueHD or DTS-HD Master Audio decoders in their home theater receivers.

With MDA, the sound engineer need not be concerned with having to redo the entire remix for multiple surround sound formats - unless they want to. MDA is designed to be a "mix-once multiple format delivery" system so a sound engineer can perform a master mix of a soundtrack and then map it to any open-source or licensed audio decoding format for distribution, if needed. In this way, both MDA-compliant and traditional source and playback components can be accommodated in an efficient and cost-effective manner.

Final Take - For Now

In the real world, we experience sound from all directions and from infinite points in space, objects produce their own sounds and we hear them. Unfortunately, in order to hear those sounds at the cinema, at home, or on a mobile device, we need to capture them, encode them, and reproduce them using speakers or headphones (headphones are just small speakers), but, unfortunately, this process is not perfect and is subject to the limitations of current sound mixing and processing techniques.

What Multi-Dimensional Audio seeks to do is provide tools that enable a sound engineer to emulate how we really experience sound either in either the real world, or in fantasy world conjured up by a content creator, which can also include the added ability for the listener to further manipulate that sound, providing a multitude of audio experience listening options. Multi-Dimensional Audio is the closest thing I have had the opportunity to experience in a home theater environment so far that captures the essence of how we naturally hear sound.

For updated details on Multi-Dimensional Audio implementation as of 2013-2014, check out an Official Announcement from DTS, as well as a detailed report by Brent Butterworth, About.com Stereos Expert.

RELATED ARTICLES

06/23/2014: Dolby Atmos - From The Cinema to Your Home Theater

08/12/2014: Dolby Gets More Specific On Dolby Atmos for Home Theater

10/17/2014: Denon and Marantz To Add Auro3D Audio In Select Home Theater Receivers

12/30/2014: DTS To Counter Dolby Atmos and Auro3D Sound With DTS:X

More From Us