Los Angeles, CA—“We have over 170 exhibitors this year, filling up the single largest hall of the LA Convention Center,” said Cosmo Scharf, founder of VRLA, in his keynote address. “Three years ago, we brought 150 people together for our first meet-up at Digital Domain.”
VRLA is just one of the many small communities that has helped virtual reality into existence, said Scharf. Thinking about the past raises questions about the future: “I’m wondering how VR will change our perception of the world,” he said, noting that VR is already driving improvements in real life, especially in the fields of medicine and education.
VR products have been in the hands of consumers—or on their heads, anyway—for barely more than a year, but explosive growth is expected. Kat Harris, technical evangelist at Microsoft, reported that IDC Research predicts there will be 76 million head-mounted displays of one type or another in the market by 2020, a seven-fold growth rate over today.
Key to that growth will be audio for VR and AR—so, despite its ‘new frontier’ status with few set standards yet, it’s up to audio pros to develop best practices and audio solutions as they develop content now.
Damian Collier, founder/CEO of Blend Media, moderating a panel on branded VR content, noted, “You can’t create ambisonic sound or spatial audio without knowing how to do it.”
Austin Mace, co-founder/CCO of SubVRsive, replied that, on a VR project for Capitol One, “The thing that really sold it was the audio. Our audio engineer really took to Facebook’s ambisonic audio tools; 99 percent of the folks that watched it, it guided their attention exactly where we wanted it, to put the brand’s messaging in their field of view.”
Sol Rogers, founder/CEO of Rewind, reported, “We’ve been doing a lot of HoloLens work recently. It has a really good ambisonic and binaural engine built into the device. Because the holograms feel physical in your world, the audio becomes so much more important.”
Thus far, VR music videos and concerts have tended to be a passive viewing experience. “Where it gets interesting for me in music,” said Cortney Harding, author, VR consultant and a professor at NYU, in response to a question from moderator Avi Gandhi, co-founder/CEO at AirShareVR, “is storytelling on behalf of artists. You can’t put a 360 camera in the middle of the stage and expect it to be compelling.” But documentary content—Chance the Rapper walking around a Chicago school he funded, or a musician explaining the creative process, for example—are much more interesting, she said.
The final panel of VRLA, moderated by Martin Walsh, VP research and development, DTS, focused on immersive audio for VR workflows. Unlike the 2D post-production process, someone working in VR must put on headphones and a head-mounted display to accurately monitor a project. “There’s a firewall between us and our workflow, and being able to see and hear [the project] immediately,” commented ECCO VR co-founder Joel Douek.
“We’ve done headphone shootouts to determine which ones have the best translation to as many different scenarios as possible,” reported Tim Gedemer, president/CEO of Source Sound. “It’s critically important to have the proper headphone environment to listen to.”
There are massive issues due to the number of different EQ curves being built into consumer headphones, said Gedemer. “We can’t possibly compensate for them all.”
“We tend to start off on speakers,” said Douek. “Then we’ll move onto headphones. The third stage is we put the head-mounted display on. Each has its limitations and advantages. As anyone who works in VR knows, it’s difficult to judge distance until you put the HMD on. That’s the last level of tweaking.”
“I’m really looking forward to the connections between digital audio workstations and game audio middleware, like the Nuendo-Wwyse connection,” said Sally-Anne Kellaway, creative director at Ossic. “I see that as the ‘golden ticket.’”
“We need to be working in AR,” said Gedemer. In the future, as the tools evolve, “We’re going to see a visual depiction of our mixing console in front of us, that’s rendered in real time. When we go opaque, that’s going to go away and we’re going to see the actual thing in front of us.”
Walsh asked if audio gets enough respect in VR. “Our job is to evangelize its importance,” said Douek. “It’s such a major part of the experience.”
In response to a question from the audience about VR location sound techniques, Gedemer revealed that Source Sound has developed a system using off-the-shelf gear to capture an effective immersive audio experience. As he pointed out, there can be no boom mics since the camera has a 360-degree field of view, and an ambisonic mic at the camera is often too far from the action and picks up too much noise.
“We’ve only got the lav to work with. We have a system made by Zaxcom; it has a built-in [recorder] at the microphone position and also transmits to the receiver,” said Gedemer. An experimental tracking system records positional metadata to reduce the panning work needed in post production.
But a lav mic picks up little of the environment, Gedemer also noted. “We use the ambisonic mic to inform us what we need to do with that lav in order to make it sound natural in the space.”