Adopting Universal Scene Description (USD)

I have seen quite a few questions around Pixar’s Universal Scene Description (USD) lately. Most seem to be about if they should adopt USD as a file format in their pipelines and workflows.

I recognize that there is some confusion about what USD actually is. So this is my attempt to shed a bit more light on the technology while I still keep exploring the depths myself. I find the technology exciting for all the efforts it’s creators have placed on finding an “universal” means of scene data exchange and management in larger pipelines.

I won’t go into too much depth as the official documentation or linked sources do that already. This article should serve as a starting point.

What is Pixar USD?

Pixar’s Universal Scene Description is primarily a scene representation in the form of an extensible graph. It has been developed in-house at Pixar, and subsequently was open-sourced in 2016. Since then it has found wider adoption in the animation industry (see below for examples).

Pixar’s Universal Scene Description is:

• Open Source under a modified Apache 2.0 license
• A scene description
• Allows to compose scene graphs
• Supports imaging (i.e. rendering) through Hydra render delegates
• Is extensible using Schemas
• Integrates with other wide-spread technologies like Alembic¹, OpenImageIO², MaterialX³, OpenSubdiv⁴, OpenVDB⁵ etc.
• And yes, it has a file format aspect, but it is more like a language describing the scene than what you would typically expect from a file format.

Scene Description

USD provides a common language to describe a scene. Part of that are facilities to define, package, assemble and edit 3D data. It allows to create asset definitions for geometry, materials, lighting, etc.

USD organizes data into hierarchical namespaces of “Prims” (short for primitive). Each prim can contain properties like attributes and relationships and are organized into layers.

USD provides a set of Schemas that establish a standard encoding and client API for common computer graphics concepts like:

• Geometry
• Shading
• Models
• Assets (referring to external files like textures etc.)

Overall, USD has all the facilities to comprehensively describe a scene in the form of a scenegraph (called a Stage) and to extract data from it.

Scene Composition

USD goes further than that though. It supports “Composition Arcs” that can be used to package, aggregate, vary, and override primitive elements and assets. Those composition arcs allows multiple artists in different departments to simultaneously work on the same asset or scene.

Each artist works on their own file (their own “Layer”).
This allows artists to work independently without editing or erasing each others work.

Composition arcs:

• Stacking using the subLayers composition arc. The composition engine will resolve the data contained in nestable “LayerStacks” similar to how layers in Photoshop are composed.
• references composition arcs allow to compose smaller units into larger aggregates, building a namespace to encapsulate the result.
• payload arcs provide a “deferred reference” that can be selectively loaded or unloaded from a Stage. This allows to create working sets that just keep the parts of the scene needed in memory.
• VariantSets allow for the bundling of different multiple variations of an asset into a single package with a variant selector that asset consumers can switch.
• Inherit and Specialize establish a relationship between a “base” and “derived” prim, so that a derived prim receives all of the overrides applied to the base prim anywhere in the composition.

The most powerful aspect of these composition semantics is that they all can be applied to any prim, in any combination, and the composition engine will resolve the resulting graph.

Stronger layers in a composition can override the scene description in weaker layers. A stronger layer can override the following:

• add new prims, including entire subtrees
• deactivate prims (effectively non-destructively deleting prims/subtrees)
• reorder prims
• add or remove variants
• override the value of schema and user-level metadata
• add new properties to a prim
• reorder properties on a prim
• override the value of any attribute
• block the value of an attribute
• add, remove, and reorder targets on a relationship or attribute connection

As you can see, the composition engine of USD is very powerful and allows for many scenarios that you need to solve in a production pipeline. USD will resolve the resulting composed scene description. That’s the true power of USD.

Render Delegates

Render delegates send scene data to a renderer and “talk” to scene delegates that consume scene data. USD allows to mix and match scene and render delegates as needed.

A renderer can be a viewport renderer (like the OpenGL based Storm renderer shipping with USD) or a production renderer.

The USD scene delegate to Hydra is meant to provide a ground truth rendering of any scene (composed of prims conforming to graphics-related schema domains). It is optimized for fast preview and animation streaming.

The major components of Hydra are:

• scene delegates, which provide scene data
• the render index, responsible for change tracking and other scene management
• render delegates which consume the scene data to produce an image

Available Hydra Render Delegates

• Pixar Renderman - a hdPrman render delegate can be build as part of USD.
• Autodesk Arnold has an open source USD integration that includes a Hydra render delegate.
• Maxon Redshift is available as Hydra render delgate that works with Houdini Solaris.
• Chaos V-Ray for Houdini has a beta render delegate for Solaris.
• Otoy OctaneRender is also adopting USD, at least in OctaneRender 2021 preview. They have implemented Hydra as “Multi-Render” that allows Octane to swap to any other render engine. There haven’t been any news in regards to USD/Hydra support since the 2021 preview announcement.
• AMD ProRender has an open source integration as Hydra render delegate. (also, see the documentation)
• Autodesk Aurora is a real-time path tracer that Autodesk open-sourced in late 2022 and also comes with a render delegate implementation.

Extending USD

USD comes with its own plug-in mechanism for the following plug-in-points:

• Asset resolution allows for custom URI-based resolvers for asset paths. This allows for custom “loaders” that e.g. support versioning, or streaming from the cloud or databases, even for procedural in-memory asset creation.
• File Formats allows to populate layers with data loaded from any file format. They can also be dynamic as metadata parameters on a payload arc are transmitted to the file format plug-in. The native formats (usda, usdc, usdz) and support for Alembic are implemented using this mechanism.
• Schemas are a very powerful concept to extend USD. They allow to add new USD prim types and APIs and work as if they were native USD schemas.

Extending USD using Schemas

There are two type of schemas that all you to extend USD:

• An IsA Schema defines a prim’s role on the Stage. A prim “subscribes” to at most one IsA Schema by setting it’s typeName. An IsA Schema is derived from the C++ class UsdTyped.
• An API Schema is an API to access a set of properties, metadata, or associated behaviors. It can be applied to multiple different types of prims. So it works like an “has a” schema rather than “is a” schema. There are three types of API Schemas:
  • Non-applied provide an API to set and get data of a prim but do not contribute to a prim’s type or definition.
  • Single-apply schemas contribute to a prim’s definition, adding built-in properties and their existence can be queried using UsdPrim::HasAPI<...>().
  • Multiple-apply schemas can be applied to a prim more than once, requiring an “instance name” to distinguish them.

Schemas can be generated manually or by using the USD schema generation tool.

Who is Adopting USD?

USD has been around for some time. It took a while for the momentum to build but looking at the adoption now at the end of 2022 it is clear that most of the animation and VFX industry is all-in. That is now also spilling over to the games industry as I see increasingly more questions from game developers about USD too.

Animation and VFX Industry

The following the main adoptions of USD in the animation and VFX industry I know of (at the end of 2022):

Houdini Solaris is one of the most comprehensive adoptions I have seen so far. It is a layout and lighting toolset that fully leverages the capabilities of USD in a very Houdini kind of way. You get a comprehensive set of nodes to compose scenes. It also uses Hydra render delegates to connect to various available render delegates.

Autodesk is also betting on USD, working on several projects:

• For Maya, they have an open-source USD integration - including support for Hydra render delegates in the viewport.
• They open sourced their real-time path tracing renderer Aurora, which is designed as a real-time viewport renderer rather than a production renderer. It is available as Hydra render delegate.
• For 3ds Max they offer plug-ins to exchange data with USD.
• At the end of 2022 they also announced an open source USD integration of Bifrost.
• Autodesk has also been looking on bringing USD to the web, by compiling the native USD toolkit to WebAssembly.

Maxon Cinema 4D is treating USD as a file format to exchange data mostly.

Blender is treating USD as a file format mostly so far. A good start. I would hope to see better support for scene composition in the future though. In the meanwhile, AMD has developed a USD Hydra plug-in for Blender.

AMD Radeon Pro Render is available as an open-source Hydra render delegate. AMD also has developed a Hydra render delegate integration for Blender which is available as open source. It includes a custom node graph to enable scene composition in Blender and also comes with support for MaterialX to exchange material descriptions with other render engines.

Adobe is integrating USD support in their Substance applications:

• Substance Stager has received USD export support in version 2.0.0 (January 31, 2023).
• Substance Painter has USD file import support and a preset to export textures for “USD PBR Metal Roughness” to USD and USDZ file formats, either as new USD file or as a layer in the original.
• Substance Modeler can export to USD as well.

Foundry always had good VFX pipeline integration as this is where most of their tools originated. So USD support should not surprise here:

• Their compositing tool Nuke has introduced a new beta USD 3D System in version 14.
• Their 3D painting application Mari has USD support with MultiShader export for Arnold and Renderman in addition to the preview surface.
• Their look development and lighting tool Katana comes with USD import and export support and a Hydra viewport. The Katana USD Plug-ins are available as open source.

The Academy Software Foundation (ASWF) ⁶ has formed a USD working group to assist adoption efforts through consolidation and sharing of best practices.

Pixar is also keeping a community maintained list of Products using USD as well.

USD itself is not (yet?) part of the VFX Reference Platform ⁷, although many technologies that integrate with USD are.

Games Industry

On the surface, USD might not be an obvious fit for the games industry. The workflows revolve around asset management. The concept of a scene does not come to mind readily in this context (other than maybe treating a level as a scene). One of the benefits of USD is that it is designed with collaboration of large teams and even multiple companies in mind. So there can be benefits of USD for asset creation and build pipelines in games.

USD is increasingly well integrated with DCC applications and the scene composition and layering capabilities allows for non-destructive workflows while keeping asset duplication at a minimum.

Studios

Remedy has been talking about their adoption of USD. They also published a version of their Book of USD which explains some of the core concepts of USD and aims at on-boarding team members.

Remedy’s main reasons to adopt USD into their asset creation pipeline are primarily DCC interoperability, asset portability (between different projects and engines) and the ability to grow content and team sizes by using composition and flexible version control system workflows. Composition and layer overrides allow them to consolidate many of their existing concepts (like levels, prefabs, archetypes and presets). USD also is performing well with large datasets.

Make sure to watch Remedy’s presentation to see how they integrated USD in both their editor and engine and how they sync updates between them using USD and gRPC.

Activision has published their Windows Shell Extension as open source. It allows for Hydra rendered real-time previews, thumbnails, and metadata properties in Windows Explorer and metadata search in Windows Search.

Game Engines

I don’t have practical experience with these game engines, so I rely on their documentation and publications to see what USD support they offer. So take this with a grain of salt, especially if you have experience with on of these engines.

Unreal Engine provides USD support through the USD Stage and bi-direction Python workflows. It does not convert USD data into native Unreal Engine assets (e.g. static meshes and materials) but offers a USD Stage Actor and a USD Stage editor window. You can import USD data as well. For more details see their documentation.

Unity announced USD support in March 2019. They provide open source C# bindings for USD in Unity that allows to import USD data as game objects or prefabs into Unity. According to their documentation the support is still experimental.

Godot Engine does not have official support for USD. Their recommendation is to use glTF 2.0 to import 3D scenes.

Other Industries

NVIDIA is building its OmniVerse platform on top of USD. That is among the strongest commitments to the technology I have seen so far. NVIDIA started to investigate USD a while ago and even provides binary builds of USD for download on their dedicated USD page.

Apple developed a set of new USDZ Schemas to extend the format for AR (augmented reality) use cases, like content anchoring, behaviors and triggers, etc. They make heavy use of USD with AR (augmented reality) and also have helped to port the Storm OpenGL viewport renderer to Metal. In their Metal documentation you can also find an example that shows how to create an application with Hydra rendering.

Further Reading

• Remedy’s Book of USD is provides an introduction to USD. It is a modified version of their internal on-boarding documentation.
• The official home of Pixar’s USD hosts an comprehensive documentation, including an introduction with leads to more in-depth topics, a FAQ, and a glossary for its many terms and concepts.
• The USD Cookbook is a collection of simple USD project files demonstrating features of USD.

Example USD Scenes

• Animal Logic’s ALab is a full production scene and is part of the Academy Software Foundation Digital Production Example Library.
• Pixar’s Kitchen Set
• NVIDIA’s Attic scene
• Intel’s enhanced version of the Sponza scene is also available as Universal Scene Description.
• Disney Animation’s Moana Island Scene is also available as USD version.