HOLLYWOOD, CA—When sound recording was first invented, it could capture many aspects of a musical performance. “But the one thing we could not record was the spatial information, so we invented reverb and echo to create the illusion of depth and distance,” says Robert “Bob” Margouleff.
Motion picture sound has delivered an increasingly spatialized audio experience ever since Disney released Fantasia in 1940, over subsequent years expanding into formats that added speakers not just behind but also to the sides and above the audience. In the 1990s, Margouleff was a pioneer of delivering surround sound outside the movie theater, remixing motion pictures for home consumption on DVD and Blu-ray Disk.
“But we were never able to deliver it in a portable way,” he says. With the recent surge of interest in virtual reality and augmented reality, and the subsequent emergence of immersive audio and VR-enabling technologies, that is beginning to change.
For the past two years, Margouleff, well known for his long association with Stevie Wonder in the recording studio, has been working with immersive audio solutions start-up HEAR360—his official title is technical audio evangelist—to fine-tune its tools. HEAR360 is the brainchild of two multi-GRAMMY-nominated recording engineers and Berklee College of Music alums. Matt Marrin, CEO, is the former chief audio engineer for Jimmy Jam and Terry Lewis, while CTO Greg Morgenstein’s resume includes work with Ludacris and Missy Elliot as well as sound design and music composition for motion pictures.
“Part of my gig for the last couple of years has been to help test their products and build new relationships,” says Margouleff. “I’m the guinea pig; I try out the equipment and see if it does what it’s supposed to do.”
Margouleff has mixed or remixed various tracks to demonstrate the company’s 12.1-channel Headphone Surround (HPS) encoder and renderer, including Stevie Wonder’s “Superstition” and The Doors’ “Riders on the Storm,” in collaboration with that track’s original engineer, Bruce Botnick. The HPS 12.1 configuration adds four height channels at left and right front and rear as well as a “voice of God” directly above the listener to the standard 7.1 layout.
Until recently, he observes, the engineer or mixer stood between the artist and the listener. “Recording music in a spatialized space with modern recording technology is not necessarily an objective experience. We create a subjective space where the music and the listener and the creator all occupy the same space.”
But whereas in the past music was typically produced in specialized environments—recording studios—people today are as likely to create content on a laptop—which Margouleff refers to as “the folk instrument of today”—with a pair of headphones. For those people, he says, technology such as HPS becomes a compositional tool.
“Why does it work in headphones? Because we can control the HRTF [head-related transfer function]. We control the crosstalk, as the headphones isolate from reality. We can create our own models of an acoustic space and tell that model where we want certain sounds to come from. I think it heightens the musical experience and brings forward a part of music that has been largely forgotten.”
It’s important to democratize the process, he believes. “I like to say that everybody is out of the bleachers and on the field, so my mission is to bring this technology to as many people as humanly possible.”
VR and AR are wholly new experiences, he says. “How we deal with the audio for these new formats is going to be very dynamic and interesting. We must make an effort to define how we move forward.”
HEAR360 first tested its live VR spatial audio streaming solution with Total Media VR in Austin, TX earlier this spring. The company’s first product, 8ball, is a patent-pending omni-binaural microphone, with companion software, that incorporates eight mic elements to capture a 360-degree sound field, and is available for preorder.
Margouleff is currently developing a number of live VR production projects, he reports. At the annual Cine Gear Expo at Paramount Studios in Hollywood over the first weekend in June, he participated in HEAR360’s demonstration of real-time streaming of VR video with 12.1 audio.
“We used three 8ball microphones with three Nokia Ozo 360 cameras from Radiant Images, which was the co-sponsor of this venture,” Margouleff reports. Radiant Images is a digital cinema company and rental house in VR and AR production.
“What HEAR360 has been able to accomplish with companies like Total Media VR and Radiant Images is a complete ecosystem, with live acquisition of program material, wireless up to the bird and down from the bird to our receiving facility. Then, utilizing our new Pro Tools plug-ins, mixing and rendering it in HPS and delivering it as streaming video in real time. And we switched between the three cameras flawlessly. There has previously been no real live streaming of VR inside one unified ecosystem; this was the first demonstration of that.”
Margouleff is excited by the creative potential of HEAR360’s technologies. “Art comes out of chaos. It’s a dynamic time—scary, beautiful, intense, violent—and we have to tell the story,” he says.
“The bottom line for me is that technology drives the art. We would not have rhythm and blues if we didn’t have electricity for the electric guitar or the Fender Rhodes piano. So we need the tools to respond in a way that we can reach people,” he says.
HEAR360
HEAR360.io
Robert Margouleff
margouleff.com
