Author: Joe D’Amato, Senior Solutions Architect, CDW Canada
Across the media and entertainment industry, the need for improved collaboration across geographies is driving the adoption of cloud environments. A significant point of discussion among film studios is whether to employ cloud, hybrid cloud or on-premises environments for their data storage.
However, turning to cloud solutions is not without its challenges, as data management costs and industry regulations significantly influence how studios leverage cloud models. These hurdles often offset the payoff from relying on cloud technologies, as traditional production practices often do not align with cloud operation models from a cost perspective.
An inability to overcome these challenges only interferes with the M&E industry’s ability to collectively evolve and take its production practices to the next level. In this blog, we take a closer look at how hybrid cloud environments can enhance rendering workflows, the common challenges studios encounter and tips for addressing these issues.
The role of geolocation
Studios have always had a challenge with geolocation between Site A and Site B. For example, a studio might have a site in Vancouver where a great deal of production work is occurring and an available render farm in Los Angeles. In legacy multisite environments, data would need to be transferred to the render farm. However, this may involve moving terabytes of data, and would require a syncing strategy that is time-consuming and includes a lot of waiting, which costs money or artist time.
To address this, many studios are moving to a single-site solution. In this scenario, studios build a data centre in between the two sites where all of their collaboration storage and render infrastructure can be stored in one location. It operates as an on-prem environment, but is collocated away from the artist.
This method has proven effective in pivoting away from site-to-site syncing. Other studios have become adept at moving files between regions to mitigate this issue. However, there are limitations to how much data can realistically be synced around the world. The next evolution of the single-site model calls for consolidating a single location within a cloud environment that houses all the data with no latency between render and disk. However, this model presents several challenges.
Encountering strict regulations
While cloud solutions can enable more agile workflows, cloud utilization practices do not always align with the regulations of the Motion Picture Association of America (MPAA) or the Trusted Partner Network (TPN). These organizations regulate how the cloud is leveraged within the M&E industry, and previous big-budget film leaks – resulting in millions of dollars, if not billions, of lost revenue – have led to tighter regulations around cloud use.
Content security is managed with the utmost caution, with studios essentially air-gapping their content away from the internet or other systems. This drives significant challenges with cloud usage, as cloud adopters often exist in a multitenant environment and lack visibility with regard to whom they share an environment. For this reason, it’s vital to verify that service providers are certified as secure by both the MPAA and TPN before deployment.
Data volumes drive high costs
Another challenge with relying on a cloud environment is the sheer volume of data and the cost of data movement. Data creation and use is significant in both visual effects and animation. This can mean, quite literally, 100’s of terabytes, or even petabytes, of data being generated daily.
When moving this volume of data in and out of the cloud, egress charges will offset any studio’s profit model. Cloud operational models are expensive and traditionally haven’t worked well for larger organizations.
Bridging the gap with hybrid cloud environments
To overcome these challenges, vendors are working on hybrid solutions to create OPEX models that bridge the gap and leverage both on-prem and cloud environments. Studios are trying to identify the tipping point between going all cloud, which remains too cost-prohibitive for larger studios, or creating a hybrid approach to strategically select work that would remain on-prem.
Dell Technologies has solutions where Dell EMC PowerScale storage (formerly Isilon) platforms are deployed into the cloud, serving as edge devices that operate with the cloud. Dell EMC ECS allows users to seamlessly move files between PowerScale and object-based storage systems in the cloud. With OneFS 9.0 we are seeing S3 support on prem and in the cloud. Additionally, some cloud providers are offering on-prem devices to facilitate a hybrid bridge to the cloud.
StudioCloud, CDW’s cloud business management software, can be leveraged to add machines to a customer’s on-prem environment over dark fibre. This enables cloud resources and OPEX expandability while mitigating data-syncing challenges. Because dark fibre goes directly to a customer’s location, and because CDW puts all our data centres near customer studios, latency or data transfer issues are minimized. Air-gapped servers and traditional bare-metal models are patched into a rack that goes straight to a customer’s data centre onsite. This approach has proven to be an effective cloud-hybrid solution that has worked well for studios.
Lagging industry practices
The technology to overcome the cloud-related challenges of cost and regulatory issues exists. While it’s true that the pricing model can still be adjusted to better suit studio needs, the most complex barrier to cloud adoption is no longer technology – it’s traditional production practices.
Traditional production practices are simply not designed to be applied to cloud. The traditional on-prem model means management has purchased equipment and management wants to see the equipment used to its maximum potential. Historically, this means artists in visual effects animation will fill a render farm as much as possible to maximize this resource, which is not the most disciplined approach. However, it’s extremely difficult to try and approach 100 percent utilization, due to scheduling issues, bin-packing and render and software inefficiencies .
When leveraging the cloud, studios now pay as they go, which is counterintuitive to the traditional “fill the farm” methodology. Instead of trying to utilize an asset to its maximum potential, studios are now switching to a model with a minimum-value proposition, with the objective being to use the least amount of farm to get a shot out. This is the opposite of how studios have operated, and as a result, many have found themselves dealing with unconstrained spending when switching to the cloud.
For example, an artist might send a render to the farm and wait until the next day to see it. When they do, they can work on it for a few more hours to make changes and then resubmit the render. When leveraging the cloud, artists can submit a render to the farm and see it in an hour or two, then work on it some more. This reduction in time enables artists to iterate far more in a day, which is optimal from a creative standpoint. However, that’s also a significant increase in resource consumption, and when paying as you go, that’s a significant increase in the amount of render resources needed while paying for those resources per core-hour. A more disciplined approach is needed to be mindful of cost.
Bringing production practices up to speed
The first practice that should be adopted to curb the overconsumption of cloud resources is a test-and-submit workflow. In this scenario, studios run test frames, have approval triggers and then move on to the bigger jobs that require the consumption.
For example, a shot may have 100 frames of a robot walking for a few seconds. To see if this shot will work on the farm, the full 100 shots are not needed; only the first, middle and last frame, which might equate to every 10th frame, otherwise known as “render on 10’s.” With a fraction of rendering, artists are able to see the result, and if it meets their needs, then submit the rest. This concept of running test frames and then running full jobs is the best way to address this production problem.
Another way is to only run small sample sizes and small, lower-resolution versions of shots until their final renders. Some of the larger animation studios are notoriously good at this practice and only run full-frame, full-resolution renders at the very end of a creative process per shot. Therefore, they’re not overconsuming resources. If artists run full-resolution, full frames all the way through a shot, they can consume 50-80 times more resources than if they took a methodical approach.
Leveraging cloud, hybrid-cloud and on-prem environments will continue to be a significant point of discussion for most studios. But the longer they take to discuss these topics and identify solutions, the longer it will keep them from employing technologies that enable enhanced production workflows, which will allow their industry to evolve.
Learn more about CDW’s solutions for the Media and Entertainment industry and watch this webinar to learn about Dell EMC PowerScale for Media and Entertainment workflows.
About the author: Joe D’Amato is senior solutions architect for media and entertainment at CDW Canada, where he analyzes and reports on industry trends, production needs, new service offerings and market opportunities, while also building relationships and opportunities in North America. Prior to joining CDW, Joe spent more than 20 years providing technical expertise for visual effects and animation companies, and the last 12 years working exclusively in the areas of render farms and storage for producing cinematic special effects. His expertise includes rendering, storage and other infrastructure issues.
