Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now


Noises off: An investigation into the developments of immersive sound

Phil Ward listens to all of the theatrical prompts listed in the immersive audio script to find out how the latest technological advancements in sound are shaping the sector

Astro Spatial Audio/ Martin Audio Sound Adventures for Kabuki Theatre

Article by Phil Ward.

Getting any sound reinforcement into theatre didn’t happen overnight. Ask Greg Clarke, Mick Potter, Bobby Aitken… now part of the creative team but not without years of lobbying on the part of The Association of Sound Designers and the pioneering example of Martin Levan, who took theatreland by the cables and plugged it in.

Immersive audio has been part of that curve for longer than you’d think, but its armoury has new weapons. Implementing them correctly is going to be no easier, and from LA to Japan there are various strategies underway to continue the struggle.

“There is a practical consideration to the speed and convenience with which you can do it,” admits sound designer Sebastian Frost, whose design for Sting’s The Last Ship using d&b audiotechnik’s Soundscape platform in 2018 won so many hearts and minds without them, in most cases, knowing why. “It’s about having the confidence that you can have a system installed to your specifications in the time available, which sometimes has budget implications. But it’s been easy to persuade people to approach the idea openly. Usually producers are keen to find new ways to interact with an audience and to get them to respond to the show.”

Frost is perfectly happy to use Soundscape’s DS100 Signal Engine with other loudspeakers in a brand- agnostic way, supporting the notion that more progress will be made the less proprietary the approach – and the greater the flexibility on hand.

Jamie Gosney is audio system designer at Stage Electrics, the UK- and Dubai-based installer, systems integrator and reseller specialising in a wide range of theatrical equipment and services. He has his own approach. “Even if other loudspeaker manufacturers don’t go down the immersive route, providers can turn to solutions like TiMax and Astro Spatial Audio and still use those brands,” he says. “I’m working closely with Out Board at the moment, a company that’s been doing this a long time without calling it immersive audio. Although it is a bit of a buzzword, I don’t think it’s going away. Mixing engineers love it, because it’s easy. They’re not fighting to squeeze everything into a stereo mix with EQ, dynamics and reverb. It’s more a question of how quickly people will buy into it.

“I’ve changed my tactics. I’ve been involved in the new sound system at the Linbury Theatre in the Royal Opera House at Covent Garden, and when we were tendering I knew we needed an edge, and suggested immersive. The technical staff liked the idea, but it all went quiet.

A few months later we got the call – and what had happened was a Eureka moment for the music director of the Opera House, who’d heard a demo. Not the engineers; the creative chief. That’s where the decisions have to be made, so now I won’t push the techs too hard; I’ll wait for the artistic leadership to get on board.”

Stage Electrics is now completing an immersive demo suite at the HQ in Bristol, where the sofas and coffee await invitations to new targets in the creative community. It’s an initiative that represents a new dynamic in the sound reinforcement industry, one not possible before: digital consoles had to win over engineers, and wouldn’t have impressed the music and artistic directors any more than chocolate sprinkles on a cappuccino. If they hear an immersive loudspeaker system, on the other hand…

TiMax SoundHub Tracker for Sankt Magarethen Opera Festival, Austria

Virtually reality

Others take the opportunities as they arise, with faith intact. “I can imagine that in 10 years’ time we’ll consider mono/stereo reinforcement systems as amusingly quaint,” says sound designer and founder/ex-chair of The Association of Sound Designers, Gareth Fry. “As the technology advances and costs come down, adoption rates will grow. I’m doing a show at the moment that can’t afford an immersive system – but it really needs one. I’ve been obsessing about imaging for many years, and if you have to do it ‘manually’, as it were, it’s far more laborious and time-consuming than using one of the new systems. But it is cheaper…

“It’s ironic. We’ve been working around these problems for a while, and we’ve come up with solutions that kind of get there, almost to the point where it can be difficult to justify the real thing that’s now available. We all know we can now do better than being clever with delays, but we still have to.

Fry continues: “Everybody has opinions, and you get lots of notes from producers about levels and all kinds of detail, but as soon as I’m in a position to use TiMax with tracking, for example, all those notes stop appearing. People stop noticing. Everything sounds so natural, they forget it’s there. Then you have the challenge of justifying a system that nobody realises is being used.”

On the other hand, some productions have no such compunctions and are beginning to promote immersive experiences as a defined aesthetic goal. If talent or audience, or both, are willing to wear VR helmets, it doesn’t seem too much of a leap of faith to accommodate 360 audio, and it does become a requisite. Fry has been working on a show at the Young Vic called Draw Me Close, with a performer in a motion-capture suit and an audience in headsets, using game-audio technology which he claims “is proliferating in the theatre industry”.

Adlib’s dearVR demo with Oculus

Liverpool-based full service company Adlib has been working with Sennheiser’s AMBEO team for some time, developing the concept alongside other immersive solutions in the hinterland where theatrical performance meets content capture and creation. “I call it ‘What You See Is What You Hear’,” says director, Dave Kay. “On the one hand, with its Neumann connection, Ambeo offers a 3D audio capture system with a quick-to-configure workflow, while on the reproduction side we can create a sound field to recreate that scalable immersive experience in a wide variety of spaces.

“But the really interesting part of the puzzle is to mix in Augmented Reality using the dearVR engine by Dear Reality in Düsseldorf, with which you can place the audio sources within the immersive space by just picking up the source and placing it where the image or the performer is within the sound field. This can be done either with VR goggles and a wand or, actually, in a 360 video playback cylinder.

“It’s more about a shared immersive experience than something on your own, and a technology that
I’m personally very excited about. We previewed the system to attendees of our January open day, and the AMBEO team is setting up demonstrations to get sound designers on board.”

The final frontier

In partnership with Dutch immersive start-up Astro Spatial Audio and launched at InfoComm in 2018, Martin Audio’s Sound Adventures package is typical of the supply now meeting demand in theatre worldwide. Yoshitsune Senbonzakura (Yoshitsune & The Thousand Cherry Trees), one of the most popular Kabuki plays in Japan, is an innovative production that combines traditional Kabuki performance with modern technology such as CG, VR, AR and motion capture. For the latest performance at the recent Cho Kabuki-supported- by-NTT event in Chiba, the production team from NATiON was keen to try some new things to make the performance better.

The original plan had been for Miku Hatsune’s vocal sound to appear from all around the venue along with the opening image video. After reviewing this, they instead decided that effects including the sound of thunder and the ‘voice’ of a dragon should be localised at the back of the venue while Hatsune’s voice continued to ‘fly’ all around. “It is the strength of Sound Adventures’ object-based solution that meant we could respond flexibly to such last-minute production changes,” says FOH engineer Mr Takahashi from Shochiku Show Biz Studio. “Sometimes we receive a request to make a 3D sound for the normal Kabuki performance as well. We’ve usually deployed channel-based solutions with a DAW, but it would be useful if this kind of object-based solution would spread further.”

“When I first listened to Sound Adventures during trials, I thought it could be used to make a more superior production than usual,” comments NATiON’s Mr Tanabe. “When I actually listened to it in the venue, I was totally convinced of the success of the show as I felt a sense of presence and impact that I had never experienced. I also heard the audience’s favourable response. I can see myself using this in various productions in the future.”

The business climate has changed, and that could mean more choice in the long run. “Immersive audio is definitely the new buzzword for something that in some ways people have been doing for a long time in a very specialized and bespoke way,” comments Autograph Sound’s financial director, Duncan Bell, “but it’s now becoming more accessible. Commercially we’re seeing some of the big names putting a lot of marketing spend into promoting their systems. Not surprisingly, some other manufacturers are thinking we’ve been doing that for years. Some of the earlier solutions from LCS, then Meyer, and Out Board were groundbreaking but very niche – and the market was small.

“In addition, use of tracking systems, where the audio tracks the performer in conjunction with the immersive systems, is an area of particular interest in theatre. The current phase of newer products means we’ll see some growth here too. But this all depends on many things: the production; the budgets; and the theatre buildings themselves, as to whether they can accommodate the multi-hang arrays, individually addressable delays and surround required to deliver the full experience. The key may be to find the way to allow venues to maintain the traditional systems but to design them with the flexibility to accommodate the requirements of the newer configurations. The ‘immersive’ word has definitely got more people talking about it, but I think it’s too early to say if this will become a must-have across the board.”

Back on track

Dave Haydon is director and co-owner of UK spatial audio pioneer Out Board, which, at PLASA last year, added the StageSpace object-based spatial environment auto-rendering tool in the new TiMax 500S software. Most sound designers have known TiMax for a long time, but as a time-delay management tool – “I’m now using it for object-based mixing too, on the back of my experience with Soundscape,” as Seb Frost reveals – and perhaps it’s still only TiMax that can satisfy the most exacting immersive requirements.

Dave Haydon

“Some people have said that certain attempts to create ‘spatialisation’ can put a veil over the sound,” says Haydon, “whereas our focus with TiMax has always been to make the system go away. Ironically, though, some people do seem to want to hear a system. Some of the latter initiatives have tended towards spatialisation that has made sure that people are aware there’s a system there, which is odd, to us, when TiMax always strives to go ‘beyond fidelity’.”

Spatial reverb has undergone a transformation, and as well as its own developments, Out Board has
an association with digital audio and sound design specialist Amadeus in Austria. “Last week, I was putting in the tracking for the Sankt Margarethen Opera Festival, an 80m wide stage and a typical multi-channel system across the front, buried in the set with surrounds,” reveals Haydon.

“The TiMax outputs that go to the speakers go in parallel to the inputs of the Amadeus spatial reverb system, which creates an artificial room for the open-air amphitheatre and responds to the movements on stage. That’s very much where we’re heading with our own spatial reverb: one that responds dynamically to the performance, as well as allowing the sound designers to dig in and be creative with it – like a post-production recording engineer might do.”

Alongside mainstream shows like Joseph and the Amazing Technicolor Dreamcoat and Mary Poppins
– which are embracing tracking with Soundscape – there is a new generation of producers really pushing immersive sound and making it a selling point. Whether that needs VR, AR or a similar techno-sell remains to be seen. But the legacy of Martin Levan survives and, who knows, maybe one day there will be a stage musical about his life. If there is, it had better be immersive.