PIXAR's Universal Scene Description (USD) Open Source

[Rodney]

This is largely unrelated what is needed by the average A:M Users but I found some of the information about USD interesting for a number of reasons.

The site provides insight into where PIXAR is heading and why some popular tech (such as Alembic) currently isn't sufficient to get them where they want to be.

 

For the A:M User with an eye (and patience) for technical explorations much can be gleaned to spice up the ol' workflow or if nothing else to capture the imagination.

 

From the site:

The key goal of the presentation at SIGGRAPH (2013) was to start a dialog that ultimately will help us decide if we should release USD and its associated IP as an OpenSource project

 

http://graphics.pixar.com/usd/

[robcat2075]

It seems like in the history of computer graphics there have been almost as many proposals to make computer graphics interchangeable as there have been computer graphics.

[Rodney]

It seems like in the history of computer graphics there have been almost as many proposals to make computer graphics interchangeable as there have been computer graphics.

 

The latest trend is a bit different in that it doesn't try to convert resources but to introduce and/or leverage open source standards and to promote, to the maximum extent possible, systems that can contain resources of any type (compatible or otherwise)*. Of course there is a limit to this and some standard must be targeted to ensure compatibility.

 

The downside of this is that we are all in our own ways very proprietary creatures and we tend to create resources that are also proprietary.

But some of this is necessary to encourage innovation.

The prime example of this is where new technology is released broadly but because plugins are specifically developed only for certain platforms.

But this also in reasonable because we wouldn't expect anyone to freely develop for a competitor's platform.

 

It is this last development element presents a real challenge to open source interchange as many new propriety states can spring into existence because the developers lack resources and time which prevents full exploration and communication within an open system. But this is also the realm of innovation where folks get to invent things over and over again (hopefully) to the benefit of all.

 

 

 

 

*This compatibility issue includes a critical element of toleration that is counter-intuitive to us because we see anything incompatible, unproven and not in line with our own personal or collective interests as wasteful and bad.

This is complicated by the fact that closed systems tend to be well known, optimized, reliable and fast.

 

PIXAR's approach here certainly isn't free of selfish ambition. They would surely prefer a proprietary solution if it was securely owned and licensed by PIXAR.

But history has proven that technology is synonymous with evolving standards and rather ironically, PIXAR is too big and proprietary themselves to innovate 'everything' themselves.

There is an interesting note two regarding USD that suggests PIXAR is committed to open source solutions at least in this specific case; their identification of the back end of USD being supported by Open Berkeley Database (OBD) which, they state, has potential licensing issues. From my perspective this appears to be a move to exert some pressure on the license holder(s) to release rights that will in turn keep PIXAR's own costs manageable and increase their profit margin. The alternative (and their negotiating leverage) is that they might otherwise be required to move to another more open standard.

 

 

There are some elements of USD that echo some plans from five or so years ago to enhance A:M's internal interchange and file handling.

The move to XML compatible file format was an early move toward that. The primary difference there and with PIXAR's USD is largely a matter of scale.

Of importance, the average A:M User already has this basic all-in-one (in-A:M) access to workflow and interchangeability.

[fae_alba]

My new job/life as an "Integration Architect" seems apprapo to the gist of what Pixar is trying to do. In my world, I work to bring multiple disparate data sources together, cleanse the data to a specific standard, then put it all together for business intelligence purposes. For Pixar, it seems that what they are trying to do is build out a solution to bring disparate data sources (pieces that when put together create a scene) and to leverage the open source community to help with that effort. Open source is a great way to generate a great deal of innovation with no cost to Pixar. What I found surprising is that they are relying on prompt based scripting to pull this together. If the goal is to offset the cost of development by open sourcing this beast, then usability should be the primary goal. Cost in these things is always measured in the number of employees required to run the system once done over the long haul. A scripted interface like this precludes that you would need a bevy of techies to maintain it.

[Rodney]

What I found surprising is that they are relying on prompt based scripting to pull this together. If the goal is to offset the cost of development by open sourcing this beast, then usability should be the primary goal. Cost in these things is always measured in the number of employees required to run the system once done over the long haul. A scripted interface like this precludes that you would need a bevy of techies to maintain it.

 

Much of this is predicated on current coding practices which are still prompt/text based. It will be interesting to see what future generations bring to the table with visual programming that automates underlying interfaces. Still I don't think we'll be seeing the end of prompt based coding any time soon as there is always a way to optimize something with ones and zeroes. The current trend into immediately being able to test/validate code live, as it's typed in, is very encouraging.

 

A:M itself is an excellent example... perhaps the premiere example... of 'code-less programming' in how it allows the creation of programs (derivatives, utilities, stories, etc. etc.) without needing to expose the user to command prompts and scripts. It's rather amazing.

 

There were several themes that have been recurring to me since reading through some of the USD documentation. The first was that of layering (non-destructive editing) as mentioned in my earlier post. (I didn't use those terms but it's there)

 

The second is that of Articulation/Rigging which PIXAR acknowledges as too 'non-standard' to standardize in USD. Their solution is a logical one... layer the rigging data over the geometry and shading while referencing the data for use in it's intended proprietary system.

 

Yet another is USD's approach to variations. Now this will be a oversimplification but A:M's Action Objects is a clear example of this and a very effective one at that. In A:M we are only a few drag and drops away from referencing multiple model variations based on the same base. Note that I consider this completely distinct from saving out a model as a new model as it then becomes a new derivative base has freed itself of any internal reference to the original base.).

 

I say all this because there is a tendency for some to read about a product and immediately think "Boy, I sure with A:M had that." when in fact it's been available to them in A:M for years.

 

 

 

In my world, I work to bring multiple disparate data sources together, cleanse the data to a specific standard, then put it all together for business intelligence purposes.

 

While I'm sure there is a lot of work involved... there surely must also be a lot of fun to be had there. Not to mention the sense of accomplishment when a plan finally comes together.

[Rodney]

This response is from one of the developers to a query about obtaining more information about USD:

we'll be making an announcement about USD at Siggraph14. We've been trying to figure out some workable ways of getting something out that people can play with, prior to our code being buildable outside of our environment. For sure, as soon as we iron out a few things in our schemas and API's, we plan to make doxygen and headers (on github) available - I expect that to be before EOY, earlier if we're lucky.

 

In the USAF I often asked my troops what they considered to be short and long term (naturally I would be following up with them on their plans for the mid term) and it seems clear that PIXAR's USD approach fills the need for their longer term goals (no surprise there).* Information may be slow to come and that in and of itself provides an opportunity to climb on board while few people have a clear view of the final outcome. This won't apply to everyone but consider... leveraging this information will be useful if you ever plan to work for or with PIXAR. Lot's of folks dream of working for PIXAR but few take the steps necessary to get there. If that is your plan... get into it!

 

As for USD, it's likely to be a slow mover in the short term as PIXAR considers how to best leverage USD with respect to short and long term goals. This translates to an opportunity for some to nibble at easily digestible information (and learn from the mistakes made by early adopters). Regardless of what you do with the information PIXAR provides, it will very likely strengthen your personal workflow

 

*But note, PIXAR isn't about to reveal all of their goals. They will surely be keeping the most important of those Close Hold.

[ypoissant]

What I found surprising is that they are relying on prompt based scripting to pull this together. If the goal is to offset the cost of development by open sourcing this beast, then usability should be the primary goal. Cost in these things is always measured in the number of employees required to run the system once done over the long haul. A scripted interface like this precludes that you would need a bevy of techies to maintain it.

What I understand is that there are two APIs: A C++ API and a Python scripting API. Of course, this means that USD is not an end-user product but requires a tech person to use it or more appropriately integrate it into a production pipeline of some sort. But that is the case for every Open Sources 3D related standards around. By themselves, they cannot do much. On the other hand, this is what makes them abstracted from one particular use case or from a small set of use cases. Rather, they can be adapted to about any use cases imaginable.

 

Reminds me of OpenEXR, the High Dynamic Range file format. Even though this is maintained by ILM, the API has evolved from numerous discussion on their user forum. The current state of OpenEXR is way more evolved than the first version and is the result of contributions from a multitude of people all over the industry.

[Rodney]

What I understand is that there are two APIs: A C++ API and a Python scripting API.

 

This is a very interesting aspect of USD. On the one hand we see binary files with all the associated speed and compactness and on the other we retain access to the file in a fully human readable form.

 

Reminds me of OpenEXR,

 

This is a good omen. OpenEXR has proven itself a worthy format and yet the surface has barely been scratched on its usefulness.

[Rodney]

Pixar has officially announced that USD will be open sourced.

 

http://graphics.pixar.com/usd/docs/Open%2BSource%2BAnnouncement.html

[Rodney]

Not long ago (and in accordance with PIXAR's plan) USD was released as open source.

The potential is there although all the various bridges aren't. (i.e. not of much use to A:M Users as present)

 

But that's not why I'm posting this.

Of more interest to me personally would be the following statement from USD documentation:

 

Hydra ships as part of the USD project

 

Hydra is PIXAR's realtime GPU renderer. (REF: http://on-demand.gputechconf.com/gtc/2015/presentation/S5327-Jeremy-Cowles.pdf)

It standardizes on PIXAR's other developmental efforts such as Open Subdiv 3.0 and USD)

 

And here is an in depth video running through PIXAR's realtime rendering process: http://on-demand.gputechconf.com/gtc/2016/video/S6454.html

It's all entirely too deep a dive into the unknown for me but intriguing never-the-less.

 

Concerning USD, it is similar to the Alembic format (which has proven to be popular) and said to be 'file format agnositc' which presumably makes it more accessible for plugin creation.

 

For those with the interest in such things more information about USD can be found at: http://graphics.pixar.com/usd/docs/index.html