New Disney 'Paperman' Technique...can AM duplicate?

[Tom]

Hello All-

 

The Academy Award winning animated short this year was called "Paperman" from Disney. They blended 2D and 3D in an awesome way.

 

Here is an explanation:

 

 

 

I don't fully understand the concept of "motion vectors".

 

Can AM duplicate this technique?

 

Many thanks for any insights!

 

Tom

[robcat2075]

That link isn't working for me but i think I've seen the clip.

 

Duplicate? No. What they are doing is a technique they've just recently invented with software they created in house.

 

When I saw it I thought that all the work they were putting into adding those lines and managing them on a nearly frame-by-frame basis seemed to be not much of a savings over animating by hand. The human input required still seems enormous.

[Rodney]

The short answer is 'yes'. Exactly Duplicate? I suppose a lot will depend on what your interpretation of what is going on there is.

 

Most people that see the demo are impressed by the ability to draw onto geometry and then have that drawing stick to the geometry while it is animated.

This is only a very small part of their technique. This aspect is demonstrated by the guy drawing the smiley face onto the hand and then the smiley face moving with the hand. In A:M, Snap to Surface is a crude equivalent... draw onto a model's surface and that drawing will stay with the geometry while it is animated. (remind me to post my tests with tattooing characters some time). But... A:M isn't a paint program so there is a limit to what can be done natively with the lines in A:M. For maximum control you'd want to post process rendered images... something Disney's approach may (or may not) be able to do in real time.

 

There are many passes that Disney's program goes through that are briefly glimpsed in the video but not explained in depth (watch the girl running up to the corner where they blur lines and use something equivalent to normal maps to layer in and refine the drawing effect). Yes, this can be accomplished in A:M but use a dedicated compositing program to post process A:M's renders and the task can be accomplished much quicker. Also, while not working in current versions there is a plugin called 'Sumi' (programmed by Marcel Bricman) that demonstrated the basic effect back in the v11 timeframe. His target was specifically 'watercolor' however and the plugin would have to be modified to achieve more control of finely drawn lines.

 

So a lot of this is guesswork at this point. If you can be a bit more specific in what you are targeting we may be able to demonstrate that same technique in A:M.

[Tom]

Many thanks for the responses!

 

To clarify...Do you think they hand animated 100 2D drawings/maps then mapped those 100 maps onto the character going through the identical action on 100 3D frames?

 

Whew...lotsa work, as you say...

 

What struck me is there is a hybrid quality to this technique that is kinda captivating..the eye doesn't seem sure if its 2D or 3D...

 

Thanks again...

 

 

Tom

 

That link isn't working for me but i think I've seen the clip.

 

Duplicate? No. What they are doing is a technique they've just recently invented with software they created in house.

 

When I saw it I thought that all the work they were putting into adding those lines and managing them on a nearly frame-by-frame basis seemed to be not much of a savings over animating by hand. The human input required still seems enormous.

 

 

If you go to YouTube you can search for -

"Paperman and the Future of 2D Animation "

 

T.

[Tom]

The short answer is 'yes'. Exactly Duplicate? I suppose a lot will depend on what your interpretation of what is going on there is.

 

Most people that see the demo are impressed by the ability to draw onto geometry and then have that drawing stick to the geometry while it is animated.

This is only a very small part of their technique. This aspect is demonstrated by the guy drawing the smiley face onto the hand and then the smiley face moving with the hand. In A:M, Snap to Surface is a crude equivalent... draw onto a model's surface and that drawing will stay with the geometry while it is animated. (remind me to post my tests with tattooing characters some time). But... A:M isn't a paint program so there is a limit to what can be done natively with the lines in A:M. For maximum control you'd want to post process rendered images... something Disney's approach may (or may not) be able to do in real time.

 

There are many passes that Disney's program goes through that are briefly glimpsed in the video but not explained in depth (watch the girl running up to the corner where they blur lines and use something equivalent to normal maps to layer in and refine the drawing effect). Yes, this can be accomplished in A:M but use a dedicated compositing program to post process A:M's renders and the task can be accomplished much quicker. Also, while not working in current versions there is a plugin called 'Sumi' (programmed by Marcel Bricman) that demonstrated the basic effect back in the v11 timeframe. His target was specifically 'watercolor' however and the plugin would have to be modified to achieve more control of finely drawn lines.

 

So a lot of this is guesswork at this point. If you can be a bit more specific in what you are targeting we may be able to demonstrate that same technique in A:M.

 

I loved that SUMI plug-in and was sorry that it is presently incompatible...

[Rodney]

Chris Oatley had this (among other things) to say about the process:

 

2.) Animation Artists Are Insane:

 

Here’s (basically) how the Paperman process works:

 

The CG animators animate the scene in CG pretty much the same way they do in any other CG film.

The computer renders the CG animation in a flat, “cel shaded” or “toon shaded” kind of way. (There’s no line art/ drawing at this point, just the fills.)

The lines are drawn by hand onto the key frames inside of what appears to be the custom software that was designed for the production.

This software “sticks” the drawings to the CG models in the keyframes and auto-generates the in-betweens the animation… Sort of.

The auto-generated in-betweens are not as artful as the completely hand-made keyframe drawings so the animators still have to go through and tweak the in-betweens to look authentically hand-drawn, appealing and organic.

The paper texture is added in compositing. …but I can’t tell if the paper texture is added before or after the drawings.

 

I could wish that A:M was a vector drawing program... boy wouldn't that be a treat... but the fact is that it isn't.

We can however duplicate the process to a very high degree.

I'm guessing the ideal way to get there would be to use A:M with Nuke (preferrably) or After Effects.

 

Here's one of the better articles I've seen on the technique:

http://www.fastcolabs.com/3006276/open-com...paperman-method

 

Don't miss the sub-links in the article (like this one):

http://www.fastcompany.com/3001775/merging...isneys-paperman

[Tom]

Chris Oatley had this (among other things) to say about the process:

 

2.) Animation Artists Are Insane:

 

Here’s (basically) how the Paperman process works:

 

The CG animators animate the scene in CG pretty much the same way they do in any other CG film.

The computer renders the CG animation in a flat, “cel shaded” or “toon shaded” kind of way. (There’s no line art/ drawing at this point, just the fills.)

The lines are drawn by hand onto the key frames inside of what appears to be the custom software that was designed for the production.

This software “sticks” the drawings to the CG models in the keyframes and auto-generates the in-betweens the animation… Sort of.

The auto-generated in-betweens are not as artful as the completely hand-made keyframe drawings so the animators still have to go through and tweak the in-betweens to look authentically hand-drawn, appealing and organic.

The paper texture is added in compositing. …but I can’t tell if the paper texture is added before or after the drawings.

 

I could wish that A:M was a vector drawing program... boy wouldn't that be a treat... but the fact is that it isn't.

We can however duplicate the process to a very high degree.

I'm guessing the ideal way to get there would be to use A:M with Nuke (preferrably) or After Effects.

 

Here's one of the better articles I've seen on the technique:

http://www.fastcolabs.com/3006276/open-com...paperman-method

 

Don't miss the sub-links in the article (like this one):

http://www.fastcompany.com/3001775/merging...isneys-paperman

 

Thanks for the fantastic links!!

 

T.

[Rodney]

I will make a rather bold statement: If you give me enough of your time I believe we can do better than what they did on 'Paperman'.*

Of course by the time we get to the 'Paperman' level of technical proficiency Disney will be well into their next innovative cycle while we'll still be perfecting our hack.

 

There are a few techniques I've personally used to draw (and paint) over characters in A:M and the primary limitation in the process isn't A:M... it's our ability to think creatively. The same techniques (in reverse) can be used to animate in A:M over any movie or moving rotoscope playing in ANY application/browser or utility. And if we can do this we can composite all the results to reproduce Disney's technique... and dare I say, even improve upon it.

 

My efforts are somewhat proprietary and I'm at the early stage of working everything out so I'd prefer the development to occur behind the scenes. I'm also a bit reluctant to put it all out there as ideas in their initial stages can be very fragile things.

 

*Accounting of course for the deficit of talent equivalency between us and Glen Keane and other highly trained and talented artists, animators and technicians.

[Vertexspline]

Rodney with the cloak of secrecy-----novel for him but so intriguing. Smiles----I completely understand Rodney everyone who is working on something new wants it to come out eventually as still being new .

[robcat2075]

... the primary limitation in the process isn't A:M... it's our ability to think creatively. ...

 

Yes. Those people we see adding lines to the CG aren't just tracers, they'd have to be capable 2D animators already to know where to put the lines and to know how they should move. That is not a trivial talent.

 

 

If we wanted to add lines over an A:M render, Flash has the tools to add lines over existing animated footage and the lines can be shaped and keyed on every frame or tweened over several frames in any combination you want. But you have to know where you want those lines.

[thefreshestever]

... the primary limitation in the process isn't A:M... it's our ability to think creatively. ...

 

Yes. Those people we see adding lines to the CG aren't just tracers, they'd have to be capable 2D animators already to know where to put the lines and to know how they should move. That is not a trivial talent.

 

 

If we wanted to add lines over an A:M render, Flash has the tools to add lines over existing animated footage and the lines can be shaped and keyed on every frame or tweened over several frames in any combination you want. But you have to know where you want those lines.

 

 

you wouldn´t wanna try that in flash... after effects would be much better suited for that. flash always messes up lines animations at some point if there´s too many cps... plus: in after effects you could add some nice effects to that lines-layer, so they would look painted.

[robcat2075]

you wouldn´t wanna try that in flash... after effects would be much better suited for that. flash always messes up lines animations at some point if there´s too many cps... plus: in after effects you could add some nice effects to that lines-layer, so they would look painted.

 

I haven't seen after effects line tweening so I can't say. My AE doesnt' have that.

 

I suppose we could to animated lines in A:M with splines set to render as lines. But this thing demonstrated in the Disney short isn't an easy shortcut to 2D look animation, it is a labor intensive way to convert 3D animation to 2D look and i don't think anyone here would want to do it for long.

[thefreshestever]

you wouldn´t wanna try that in flash... after effects would be much better suited for that. flash always messes up lines animations at some point if there´s too many cps... plus: in after effects you could add some nice effects to that lines-layer, so they would look painted.

 

I haven't seen after effects line tweening so I can't say. My AE doesnt' have that.

 

I suppose we could to animated lines in A:M with splines set to render as lines. But this thing demonstrated in the Disney short isn't an easy shortcut to 2D look animation, it is a labor intensive way to convert 3D animation to 2D look and i don't think anyone here would want to do it for long.

 

which ae-version are you using???

 

of course you´re right... no one here would spend hours on refining a few frames

[Rodney]

One of the things that a sketch tweening plugin would likely have to do is compare the CPs in two 'models' and then assign the same (or similar) CP Number or Name to both of them. The splines could then be tweened to make a third inbetween by dividing the distance between the two. This is doable in a sterile setup that is highly controlled but the variables of how people draw are entirely too many. From time to time I do think of plugins such as Steffen's 'Connect' plugin that are designed in such a way as to associate CPs together so I know the process is programmable.

 

One experiment I've long wanted to try but haven't yet attempted is to find a way to constrain CPs to a 2D plane in front of the Camera in a Chor. This is very doable in a Model window via Snap to Surface so it should be doable in a Chor too. The idea there being that spline drawings could be written like annotations over the top of any scene or setup. We can probably get this done now just by locking everything down as then A:M wouldn't 'see' the geometry so it would simply drop the CPs onto a 2D plane in the Chor. (Come to think of it I believe I did test this enough to satisfy my curiosity that it would work)

 

Edit: I just refreshed my memory about some of the annotation tools I was developing a few years ago. In a Chor the splines will fall onto the Camera Plane if all of the Objects in a scene are set to Unpickable. This is quite useful and save us the step of placing a 2D plane in front of the camera to draw on.

 

One thing that would likely need to be introduced would be a mode where CPs and splines could be drawn without constantly clicking into a window. Holding a key down while splining might suffice for an acceptable spline drawing workflow (i.e. holding the Shift key down while moving the cursor would tell A:M that it is in 2D drawing/annotation mode.

 

Later, as necessary, that 2D drawing of splines could then be projected onto any object via Steffen's other plugin that allows CPs to be snapped to objects from any angle. If you haven't used that before... you should!

[robcat2075]

which ae-version are you using???

 

I got my AE 5 in 1999. It has a "Paint" but there's no tweening of lines like Flash does.

[thefreshestever]

which ae-version are you using???

 

I got my AE 5 in 1999. It has a "Paint" but there's no tweening of lines like Flash does.

haha... ok, that´s freakin old!!! it does have that now... among a few other things

[robcat2075]

I did the one-month trial of the current AE recently but I was disappointed it wasn't any faster at working with footage than the one I have now.