Lip sync?

[gra4mac]

Hi. I'm working on a lip sync segment and I have a question on how others do the syncing. I stared with 3 +/- mouth poses, smile/frown, open/closed, and ooo/aaa, then used these to make the 9 Preston Blaire phonemes. After creating the dope sheet, I found it took a lot of tweeking to get it looking good and I got realy confused trying to adjust all the channels. What I ended up doing is not using a dope sheet and just animating to the sound track using the 3 basic mouth poses. This worked better and was much faster; lesson learned.

 

I am just wondering how others do their lip syncing.

[KenH]

I'm humming and hawing over which system to use to animate my characters face. It's the first "serious" character I've ever done and your system sounds like an elegant one. Do you find with those 3 sliders, you can get all the basic shapes of a mouth?

[gra4mac]

I found I got a satisfactory range of mouth movemant. I am not doing photoreal stuff so close is good enough. I don't think you could lip read it. I hope to have a clip to show later today. I did a quick render test and the 3 pose method rendered about 15% faster than the PB phoneme version.

[ZachBG]

Dope sheets aren't worth the time it takes to fix them. My lip sync contest entry from last year was entirely done in the method you wound up with, Graham.

 

(still meaning to do a tutorial...)

[gra4mac]

Thanks for confirming that this works ZachBG. I feel so pleased with my self for figuring it out. I need to tell somone else. Hay Mom! guess what I figured out... No not that kind of dope. Never mind.

[SeanC.]

I use Jason Osipas facial setup. I love it! ITs a long list of mouth shapes but its well worth it. He uses visemes, shapes you can see rather than phonemes, shapes you can hear. Some of the poses are mouth open/closes , upper lip up/down, lower lip up/down, sneer, ooooo, pucker, narrow, wide. All of these are of course split into left and right sides of the face. The there are different brow shapes. I recomend buying his book, "Facial Animation and Modeling Done Right" It really helps.

 

SeanC.

[heyvern]

Yeah, this is a toughy.

 

Personally I have done it both ways. I used the dopesheet technique for a project from last year a group from the list put together (still in progress?).

 

Fractured Film Project

 

It is the very first part with Thom and that bear guy from the CD.

 

For me in that situation it worked great and it seemed to go pretty quickly (time was an issue). I was able to rough it in and then tweak it. I really liked using the dopesheet, especially when I could just type in the dialogue to get the first rough keyframes for all of the words. The tricky part that was actually kind of fun was coming up with wierd alternate words to match the dialogue.

 

But as you have already found the best technique is the one that works for you.

 

Vernon "Sheet! You dope!" Zehr

[rbjohnsn]

What is a comfortable length for the dopesheet? I've got a somewhat lengthly diolog, and it seems like the memory locations are very eractic. I've tried the problem on three different computers each using different verions of XP, (Home, PRO, Media), each exsibet the same issue.

[HomeSlice]

What is a comfortable length for the dopesheet? I've got a somewhat lengthly diolog, and it seems like the memory locations are very eractic. I've tried the problem on three different computers each using different verions of XP, (Home, PRO, Media), each exsibet the same issue.

I've only used the dope sheet maybe 10 times, but from what I can remember, it seems to favor fairly short snippets - no more than a short paragraph. So create several Actions for your dialog and line them up in the Chor.

[rbjohnsn]

What is a comfortable length for the dopesheet? I've got a somewhat lengthly diolog, and it seems like the memory locations are very eractic. I've tried the problem on three different computers each using different verions of XP, (Home, PRO, Media), each exsibet the same issue.

I've only used the dope sheet maybe 10 times, but from what I can remember, it seems to favor fairly short snippets - no more than a short paragraph. So create several Actions for your dialog and line them up in the Chor.

 

Thanks for the suggestion.

I have not had a chance yet to try a dopesheet for each sentance. What I've seen buy looking at the ACT file with a tex editor is the fram counter gets reset to zero for the word. I beleive it is a math function, of the software using the mouse, to position the word with the WAV file. The dopesheet also gets corrupted when playing the action. I've played with this operation now for 6 years and have just begun to understand how to manpulate the controls. The video tut helps, but still very vage as to how.

[HomeSlice]

Unless you don't mind choppy and abrupt lip movements, it is actually faster and easier to do it manually instead of with a Dope Sheet.

First pass - just open and close the mouth to the sounds.

Second pass - do the "OHs" and "Ws"

Third pass - so the EEs and SSes

Do more passes if you have more poses, such as F, V and L.

 

Look in the Channel View and make sure all keyframes are using Spline interpolation. Sometimes they use Zero Slope even though the default is set to Spline. Then switch the interpolation of certain keys if you need more crisp movement in places.

 

Except for very fast or slow bits, open/close movements are usually 2 frames. In most cases, start the movement 2-3 frames before the sound, so it ends on the sound or 1 frame before.

 

Ohs, Ws and Ls are usually 4 frames. EE, SS, F and V - I usually time at 2 frames.

 

If you have to go 1 frame because of very fast speaking, only change the mouth position a tiny bit to avoid unnatural choppy action.

[robcat2075]

Unless you don't mind choppy and abrupt lip movements, it is actually faster and easier to do it manually instead of with a Dope Sheet.

First pass - just open and close the mouth to the sounds.

Second pass - do the "OHs" and "Ws"

Third pass - so the EEs and SSes

Do more passes if you have more poses, such as F, V and L.

 

This is basically what they teach at AnimationMentor so Holmes just saved you a couple thousand .

 

In real life mouth movement is pretty subtle. In animation, things like B K or a hard M at the beginning of works may benefit from punching them a bit harder.

 

To speed up roughing in the keys you can make drag and drop poses for the various mouth shapes. See tut on drag and drop poses in my tuts link.

 

the real hard part of lip synch isn't the lips, it's the body language. i f you get the body language going you can get by with surprising little polish on the mouth.

 

 

If you really want to read a lot about lip synch, the Jason Osipa book "Stop Staring" covers his approach to animating facial features. The first edition even mentions A:M although he was stuck on v8 at the time.

[mouseman]

I wikied HomeSlice and Robcat's advice for posterity.

[HomeSlice]

Awesome mouseman. Thanks.

[Rodney]

Holmes description is one of the best and most concise I've seen to date.

 

Here are some additional tips/general rules/suggestions regarding dialogue phrasing passed on from other animators (Art Babbit/Milt Kahl/etc.) via Richard Williams:

 

Open the mouth on accented vowels.

Close the mouth on primary consonants.

Don't reopen the mouth (during a continuous phrase) without good reason. (i.e. keep it open throughthe next vowel sound and close it on the next major/hard consonant)

On emphasized words vowels can pop open (i.e. don't inbetween them) and they occassional can pop closed.

When animating the tongue, don't inbetween it's movement (it moves too fast for that subtlety of movement).

Animate the body first. Ideally the audience should believe the dialogue was given by the character even if their mouth doesn't move. (The example is of Disney animators that added the mouth animation after all other (body) animation was complete.)

 

Richard Williams suggests the secret of animating dialogue is to progress the action of your character during/throughout the dialogue(whatever their action is in the scene). The character must be going somewhere... doing something... considering something. (What are they thinking? How does their action fit with or differ from what they are saying?)

 

Rather than syncing an action to individual words we tend to blend words together when we speak... so a direct transliteration of the wording is usually not desired. Locate the emphasized vowels (usually 'O') in a phrase and focus on those in your character animation. If we progress the action along with the emphasized moments in a phrase we can use open/close muppet-style posing and the dialogue will be convincing... as if the animated character had spoken the words.

 

 

(Note: I'm paraphrasing, am missing some points and have added some of my own interpretations here)

[rbjohnsn]

Thanks all for the suggestions, but being the tanatious person I completed the whole dialog with short sentences. Had it working pretty good until I tried to get the clothing simulation to work again. When resolving that issuse I somehow lost all the phoneme breakdown. It turns out, it will only function when adding a new word, sentence, etc. .. Question: is there any way to redo the Auto Pheneme Breakdown on all the diolog currently in the Dopesheets without having to start over reentering each word again?

 

Thanks for any help!

[Rodney]

Question: is there any way to redo the Auto Pheneme Breakdown on all the diolog currently in the Dopesheets without having to start over reentering each word again?

 

It's been awhile since I used the Dopesheet but... I thought the phoneme breakdown was saved.

Is this not the case?

 

Of course if something has removed or replaced that you'll either have to copy from a saved version or relaunch the Auto Phoneme.

Have you tried Right Clicking to see what options are there?

[rbjohnsn]

Question: is there any way to redo the Auto Pheneme Breakdown on all the diolog currently in the Dopesheets without having to start over reentering each word again?

 

It's been awhile since I used the Dopesheet but... I thought the phoneme breakdown was saved.

Is this not the case?

 

Of course if something has removed or replaced that you'll either have to copy from a saved version or relaunch the Auto Phoneme.

Have you tried Right Clicking to see what options are there?

 

I did recopy from an older version. Thats when I lost the sync in the action file. The "Auto Pheneme Breakdown" funtion is grayed out preventing any futher occurence of using it.

 

This all happened because there was just one frozen CP in my cloth that was causing the simulation error. I recovered the cloth simulation by using Beta verion 16 step 5. This has be another learning experince with AM!