This year’s AES Convention took the theme “Immersed in Audio,” a play on words that described the experience of losing oneself in the new technology on display while also encompassing the new conference-within-a-conference focusing on the immersive worlds of Virtual and Augmented Reality. The two-day AR/VR Conference, which required separate registration to the 141st AES Convention, was also located within the Los Angeles Convention Center and featured workshops, tutorials and a companion product expo.
VR has certainly been grabbing headlines and the attention of consumers. But while its potential impact is great, offering applications far beyond games, motion picture spin-offs or travelogues, VR is still in its infancy and is far from reaching critical mass just yet. And for the audio industry, VR represents the new Wild West—so it’s perhaps appropriate that the inaugural AES International Conference on Audio for Virtual and Augmented Reality was launched in the former American frontier state of California.
As a presenter at a recent video game audio and sound design conference in Los Angeles jokingly remarked, if anyone tells you that they are a Virtual Reality audio expert, run the other way. Sure, the platform may have seemingly sprung out of nowhere, but the underlying science is decades—even centuries— old. The current iteration of VR is certainly only a few years old, yet a good number of audio professionals have been grappling with immersive audio workflows for quite some time, especially those working in the video game field. Happily, some of those practitioners were on hand at the AR/VR Conference to exchange ideas, share their valuable experiences and attempt to resolve the many and various audio challenges presented by VR.
The program, helmed by conference co-chairs Andres Mayo and Linda Gedemer, provided an overview of the AR/VR creative process, applications workflow and product development. The event also provided an activity hub, focusing attention on the Audio Engineering Society from the expanding creative communities that are working in an area of endeavor that could perhaps eventually benefit from the guidance of a standards-making body such as the AES.
The conference’s companion expo featured a handful of manufacturers and service providers offering solutions to this emerging field, complementing a number of relevant products also on display on the main convention floor. Those solutions included products for two critical parts of the workflow: microphones, and post production and rendering software.
Sennheiser unveiled its strategic focus on immersive audio at the beginning of 2016, showcasing its 3D audio Venue Modeling software, Ambeo 3D audio listening experience and Virtual Reality microphone at conventions throughout the year. According to Sofia Brazzola of Sennheiser Strategic Innovation, “Within the framework of our Ambeo 3D audio program, we have been active in the VR production field for quite some time, developing the optimum solution for immersive audio capture.”
At AES, in addition to participating on a panel, “Immersive Sound Capture for Cinematic Virtual Reality,” Sennheiser highlighted its newly available Ambeo VR microphone, which is fitted with four matched KE 14 capsules in a tetrahedral arrangement. The mic utilizes one of VR’s decades-old foundations, ambisonics, to capture a spherical soundfield in four quadrants and then output it in B-format, enabling post production manipulation of the audio image.
The original soundfield mic was introduced by Calrec Audio back in 1978; the current SoundField mic is available from TSL Products.
VisiSonics presented a VR capture solution that is both a microphone and a camera. More specifically, the VisiSonics 5/64 Audio/Visual Camera comprises five video cameras and 64 microphones mounted on the surface of an eight-inch diameter aluminum sphere, with proprietary algorithms enabling the creation of an omni-directional acoustic image, with synchronized images.
The PCM audio is captured via an internal FPGA—a dynamic platform that will support future iterations of the product—and output in a single USB 3.0 data stream. Software allows the acoustic image to be manipulated in a variety of ways, including presentation in filtered planewaves or beamformed into multiple real-time beams. The technology reportedly doesn’t come cheap, at roughly $1,000 per mic element for the system.
The biggest hassle for anyone working in VR audio has been the lack of a cohesive suite of software post production tools. Facebook threw its hat into the ring earlier in the year when it acquired Scotland- based developer Two Big Ears, subsequently making its rebranded Facebook 360 Spatial Workstation available for free download. At the 2016 AES Convention, Dysonics joined the fray with its Rondo360.
The application is said to run in all major DAWs as a plug-in (VST, AU or AAX) and “can support all channel configurations, regardless of what your DAW supports,” according to Dysonics. The software incorporates ambisonics encoding and decoding, as well as 3D audio and surround sound spatialization. Output is to Dysonics’ proprietary MTB (motion-tracked binaural) format for head-tracked headphone listening.
Coming soon, according to the Dysonics web site, is the ability to live-stream from ambisonics microphones, the company’s patented RondoMic array or multiple mono and/or stereo mic setups.