Give The Audience What They Deserve By Brad Berridge

From the very first sound system deployment around 1915 and countless others since, we sound people have made a compromise.  In an ideal world we’d put a loudspeaker in front of each sound source, be that presenter, singer or instrument. In practice of course this would ruin the visual picture and detract from the physical presence of the source.  Since the idea of an amplified speaker helmet for everyone never caught on; and since by nature all sound people only want the best for every production and event audience, loudspeakers were moved to the sides of the stage. With that compromise made, we have been looking for ways to overcome its spatial consequences ever since.

We’ve gone along for a good many years now coming up with solutions to fix what we have termed “stereo” or “multiple mono”. While a great time can be had with a well-tuned system and your favourite band or theatre performance, stereo for most of an event’s audience will not work effectively.  A typical stereo experience is created by the physical nature of the sound delivery system and our perception.  Sound comes from a pair of speaker channels with the audience usually within the field of both speakers.  We are all familiar with the “sweet spot”, or that central location at which the two signals converge to give you a balanced image from both channels.  This is typically only a very small percentage of the total coverage area – usually also hogged by FOH at a live gig!  So, relatively few in the audience get a stereo experience because no matter where they’re sitting or standing in the venue, sound will arrive from one speaker channel or the other with slightly greater intensity, but most significantly, it will be earlier or later in time.

Multichannel spatial mixes using distributed systems can be achieved to some extent by spreading out more than just the usual two stereo speakers, so that different parts of the mix come from each speaker. The attempt is to cover everyone using all the speakers with a better “stereo” mix.  However, because coverage from all these channels will vary, and again, arrival times will be substantially different, achieving this effectively can be challenging. But we’ll come back to this…

Stereo is a myth

From the first word performers utter, to the downbeat of the band or orchestra, the audiences’ brains have work to do to fix our conventional stereo system compromise. The brain knows the sound is coming from the left and right speakers in a “stereo” configuration, but it localises them by both level and time differences in arrival at our ears. This is how we evolved to survive, knowing exactly where sounds of danger were coming from. When something is on our left, it arrives in our left ear first, travels around our heads, and then into our right ear. It’s called the Haas effect: The brain actually discerns the location of a sound, not so much by level as the differences in time of arrival in each ear.  

In our stereo setting, the audience can plainly see the performer singing or delivering lines downstage centre, but their perception of the amplified version depends on whichever speaker channel they are closest to. The brain says, “This is not ok, our eyes tell us what we are supposed to be hearing but our ears tell us different.” 

Of course, we could give the audience headphones to create a truly stereo experience, but this is impractical on many fronts – as well as isolating – so we dismiss this as a practical solution in most cases, other than a few specialty immersive theatre or experience instances.

Some of the best productions I’ve attended were small showcase productions without sound reinforcement. Not a microphone in sight! A piano on a small stage, actors nearly in your lap; no more than twenty to thirty feet from every audience member. When the piano player hits the one on the downbeat, the sound comes from the piano; when one performer sings stage left, that sound comes from them. When another performer speaks from upstage right, our heads turn to locate them.

While everyone hears things naturally localised based on their physical location in the room, everyone is getting a full, shared, sonic experience. Each of the sounds come from the originator of that sound – wherever that is in our space. It removes the extra step our brain must do when there’s a conventional PA system, by eliminating the fact that the full belt vocal solo that brings everyone to their feet appears to come from the speaker array on one side of the stage because the tickets we bought are slightly right of centre.

Switzerland’s Thunerseespiele 

What if we abandon the idea that stereo is actually possible, or that it should be our default set up for a performance or event sound system?  We now have the technology to fix the problems created from that initial compromise. We now have spatialisation and object-based mixing engines and software to deliver a more realistic and universal experience to our audiences.

Using the principles of the Haas Effect delay-imaging previously mentioned above, managed by an object-based, dynamic level and delay-matrix platform, new strategies and techniques – coupled with just a few more speaker channels – can make the amplified sound come from its physical location onstage to deliver a meaningful, truly united experience for our audiences.

The principle of these systems involves using a distributed system of speakers – whether line array or point source.  The individual signals are “placed” spatially in a virtual stage that represents the physical one (in TiMax these are called these Image Definitions), then the spatial audio processor applies the correct delay timing and level shading to the individual input sources so that the whole audience perceives the source coming from its location on stage.

Some vendors attack this with simple level-panning and some with classic wavefield synthesis. A few, such as TiMax, utilise automated object-based dynamic delay-matrix rendering with highly-granular parameter control. This results in greater spatial mix coverage with often less critical speaker system designs.

Sound comes from where we see it coming from.

Tool World Tour 2020

The audience does not need to go through the mental exercise of knowing where the sound is supposed to be coming from. They can simply hear without this barrier.

This paradigm shift can introduce a new era of more critical listening and a perceived “bespoke” experience for every audience member.  The show will sound more spatially consistent in every seat, the experience will be universal. Everyone will know where the sound is coming from.

Beyond frontal systems for concerts or presentations, these new spatial platforms can also introduce mix engineers and sound designers to powerful tools for the control of 3D surround sound, transparent theatrical effects’ systems, and immersive experiential installations. All systems can benefit from the tactical use of delay as well as level for a more intimate immersive experience. The only limit is our imagination.

When I was planning on deploying one of these systems for a large-scale theatrical production, I brought the directors into a demo of the system so they could hear it, and we could begin to establish the way we would address things artistically. After just a brief demonstration of object-based mixing on a frontal system and a short presentation of what we could accomplish, they were hooked. One of the directors remarked “I am now excited to use sound in the show. It was always the last thing I would deal with because it was the least fun.”

Back in 2016, my wife and I attended an arena spectacle show. We had decent seats, just off-centre on one of the long sides, about halfway up in the 100 section. It was a great place to see everything since the show utilised most of the arena floor. After the show, I was grilling her on what she thought of the show (as our partners are used to, I am sure) and she was frustrated. I pushed her on why and she said “I never knew what was going on-I felt like I missed the first line of every scene trying to find where to look. You must fix this!”

Fortunately, with the right spatialisation and vocal localisation tools and techniques, we can fix it!

When I subsequently got the opportunity to design a theatrical show utilising one of these spatialised, object-based mixing engines, it was a much more enjoyable experience both for my wife and the entire audience. I could guide their eyes and make the music and action come alive. The music sounded better than it did coming out of a “traditional” system whilst the vocals were clear and actually came from the performers producing them!

Mamma Mia! The Party at London’s O2 (Photo by Luke Dyson)

As we have seen with other advances such as line arrays, DSP, digital consoles, software audio playback etc., the realities of “new” technology deployment can change to favour the audience quickly. It happens as technology develops further and more of us demand it on our productions.  Within the next decade, object-based audio will be the norm rather than the exception. This is what our audiences deserve.

It will unite them with the stories we tell in a more intimate way. Let’s work to make it happen.

Read more on the science of spatial audio via the link below.

https://timaxspatial.com/support/science-of-immersion/