Moore’s Law was invoked several times at the recent eighth annual Creative Storage Conference in Los Angeles. But as noted by conference participants, storage has lagged behind processing power, leaving a performance gap that has really only started to close over the past decade.
As Intel’s Gordon Moore observed in 1965, semiconductor processing power doubles—or the cost is halved—every two years, although some argue that’s now closer to 18 months. Naturally there is an equivalent law for storage, named after Mark Kryder of Seagate, who noted in 2005 that magnetic disk storage density now doubles approximately every 18 months.
One current hot topic in storage is collaborative workflows, with data being moved over dedicated networks or the internet. But for all the talk of the cloud, local hardware still reigns supreme.
“Connected storage is growing fastest because it enables people to work together and get jobs done faster,” according to conference founder and industry consultant Tom Coughlin, of Coughlin Associates, in his opening address about storage trends in the media and entertainment (M&E) industry. Coughlin noted that direct-attached storage at individual workstations continues to grow steadily.
One factor driving collaboration is storage interfaces, which are becoming increasingly standardized and offering faster and faster throughput. PCIe interfaces in general are driving the industry, he noted: Thunderbolt, for instance, is at 20 Gb per second.
That said, local physical storage media is starting to be supplemented or replaced by cloud-based services, which are also enabling collaborative workflows. “Content is moving in the cloud; this potentially has a very significant impact on the media and entertainment space, just as it does in traditional enterprise environments,” said Coughlin.
“When you talk about the cloud these days, it’s not only about the Internet of Things,” said Elaine Kwok of Promise Technology, referring to the phrase coined by Kevin Ashton of MIT’s Auto ID Center in 1999, “but the Internet of Everything. Everyone seems to want to be ‘cloudy.’ They start using words like ‘anywhere’ or ‘everywhere.’ The question is, where is it?”
Kwok noted that there are several cloud-based collaborative workflows being offered, especially for video editing, including Adobe Anywhere, Avid Everywhere, MediaSilo, Aframe and Forbidden Technologies. “Avid’s more about breadth,” she said. “It includes not only editing but also monetization and distribution, versus Adobe, which is more about depth. [Adobe is] more specialized in video editing specifically rather than encompassing the entire workflow.”
Moderator Philip Hodgetts of Intelligent Assistance asked his panel how a small post house can best connect to the cloud in order to make virtual workflows practical. “You need [a] 10 gig [Ethernet link] and you need some kind of GPU accelerator, like Nvidia Grid, to properly use a cloud-based workflow. Otherwise you’re shipping files back and forth,” said Shane Archiquette of Hitachi Data Systems. “You don’t want that running voice and other internet traffic; it should be fairly dedicated.”
Archiquette cautioned that the efficacy of cloud-based workflows can be geographically dependent, as latencies can vary tremendously depending on proximity to the data centers. “Within metro L.A., it’s zero to 7 milliseconds,” he reported. “Once you go outside [the downtown area], you start to hit 13 ms. In Phoenix, you hit 26 ms. Understanding where media workflows can work within those latency ranges is what really matters.”
A panel of Hollywood post house technologists considered the current state of storage. “You try and move data as little as possible,” said Brandon Bussinger of Working Order. But productions are going worldwide, chasing tax incentives, he noted, and they typically don’t have an IT team. “Every file we create does not create one workflow problem; it creates about 10,” he said.
“We need help doing the things that humans are really bad at—remembering and keeping track of details over time,” said Josh Rizzo of Hula Post. The greatest challenge is the lack of a metadata standard, he believes. “We are inheriting other people’s tech. The future is grim if there are not some standards for the products offered to the entertainment industry specifically.”
There is some hope on the horizon. As noted by an audience member, SMPTE is working on developing a Core Metadata Standard currently, based on a subset of the EBUCore data set.
What storage is used in which segment of the workflow, asked another audience member. “We’re just getting our feet wet with object storage,” said Brian Kenworthy of Deluxe, “to virtualize the workflow.” Ingest and capture is still on SAN. “Anything in-between is still done on NAS.”
According to John Stevens of MTI, “We’ve been testing object-based storage, primarily Ceph. It’s definitely on the roadmap; it’s driven by price. That’s the big promise.”
“We’ve actually created a service out of digital archive preservation,” Kenworthy reported. “Storage is our third top cost in our entire company. We’re always looking for ways to decrease that.”