Lip syching smoothly

[heyvern]

Only the upper TEETH don't move. The lips are all rubbery and move all over the place. Also depending on the REALISM of the character you can "cheat" or stretch "reality" and move the upper jaw to create exaggerated poses.

----

I suppose the AS Pro demo doesn't have any of the content provided correct? I was going to suggest looking at the "Fritz" character. He has head turns and interpolated mouth shapes for lip sync.

-vern

[mkelley]

Fritz is there -- it appears as if all the content is there (interestingly enough).

[etherpictures]

You really can't. I would just try not to move the center of the top of the mouth to much. Great thing about animation is that you get to break all the rules. Also for U and Q I tend to just copy the O make it smaller and move some of the sides out. With the scale tool you end up having to "push" the whole shape back up a little so it's harder to get it exact.
I hope you have enough points. If it turns out you don't have enough to make an O, I would start with the hardest one to make, like L or something, and use the split curve script on the outer shape and pick just like 1.

[turn]

would it be possible to have smooth lip syncing that is automated? or does automated lip syncing have to be choppy looking?

[mkelley]

Automated how?

If you mean using software like that in AS 6, which reads an audio track and attempts to match up phonemes to text you've typed in, then the answer is yes, mostly. The phonemes will indeed interpolate smoothly (as long as you've designed the individual layers correctly and turned on interpolation).

The problem is that "auto" lip sync is about as good as "auto" anything -- there's no magic button you can press to animate your character, and there's no magic formula to get good looking lip sync. The auto function in AS 6 is about as good as it gets, which is to say, not very.

Lip sync is an art, not a science -- look at the credits for any animated series and you'll see there is nearly always a team or at least one dedicated person JUST to do the lip sync. For one thing, not everyone forms the same phonemes when they talk -- some people talk out of the side of their mouth, some have accents, some don't open their mouths very much, etc. etc. For another thing depending on how fast the dialog is you may need to move the track forward and back at times to get it to look right even if you've read the track absolutely perfectly -- Disney animators were the first to notice this and it's amazing how sensitive the human eye is to bad sync -- even a frame off looks odd at times.

So the real bottom line is that if you want it to look right there is absolutely no substitute to doing it by hand, the way animators have done for decades. There are tools that make it easier -- Papagayo is still one of the absolute best ways to set up your track and it does automate to a large degree the reading of the phonemes in the typed text but it is still a time-consuming process. Or you can just use the auto function in AS 6 and not care how it looks -- it's up to you.

[heyvern]

So the real bottom line is that if you want it to look right there is absolutely no substitute to doing it by hand, the way animators have done for decades.

I agree with Mkelly that doing lip sync by hand is the best way but is also the most time consuming. If you have short dialog sequences doing it by hand would not be too bad. If however you have to or want to produce a "series" each week and you have limited resources (uh... you do it all yourself ) those tools for "automating" lip sync can do the job well enough that it doesn't look so bad.

This sample below was done VERY quickly using Papagayo. I did this ages ago when I was just learning "Moho". It is done using inerpolated switch layers. Making all the layers interpolated was harder than creating the lip sync. The mouth moves pretty fast when you're talking so the interpolation can often be wasted effort.

http://www.youtube.com/watch?v=GLNzgF_K8YM

Papagayo is a great tool and, yes, it can take a bit of fiddling lining up the phonemes to the audio. Still, it's faster than doing it by hand. The auto lip sync with switch layers in AS 6 is even faster but still is going to require some tweaking.

At the end of the day you have to look at your schedule and/or budget. If you don't have the time or budget to do lip sync by hand and the quality of auto lip syncing looks okay (it isn't "Disney" or "Pixar quality but it's fine for us mere mortals) then use it.


-vern

[mkelley]

Vern,

I didn't mean that doing it by hand doesn't preclude the use of PG -- that's what I use for all our dialog. But PG is hardly automated, as you have to slide each word around on the timeline (and even phonemes depending on how your characters talk).

I assumed by "automated" the poster was asking about something like the auto lip sync in AS 6 which is dreadful, IMHO, for all except the very shortest of dialog (like "Hello world!") and even then PG would be better and not that much more difficult to use.

I've read tracks by hand in the past and PG is absolutely the best software tool I've ever seen to automate that dreary task but it's no "auto" solution. For our series it still requires days of work for each show to get the dialog right (then again, we have LOTS of dialog -- talking heads, basically, as I have no animation skills).

(And that's a nice piece of animation and lip sync -- was that Kath Hepburn doing the dialog?)

[heyvern]

I assumed by "automated" the poster was asking about something like the auto lip sync in AS 6 which is dreadful, IMHO.

Dreadful? Really? Dreadful? <sigh>

(And that's a nice piece of animation and lip sync -- was that Kath Hepburn doing the dialog?)

No it wasn't Kate. I forget the actresses name. It was a clip from the old movie "Shop Around The Corner" with Jimmy Stewart. I had been only using AS for a few weeks or maybe a month or so when I did that. I had just started using Papagayo and it saved me HOURS of hand keying lip sync. I did 8 minutes of dialog in just a couple of days.

-------

So the real bottom line is that if you want it to look right there is absolutely no substitute to doing it by hand, the way animators have done for decades.

I wasn't disagreeing with you about the quality of hand keyed lip sync. I was merely pointing out that the above statement might scare beginners off from doing something easier to suit their schedule and resources and still end up with fairly good results.

-vern

[mkelley]

Hey, I qualified my opinion of "dreadful" (IMHO). Just me -- YMMV. (But I did run some pretty extensive tests and based on the stuff we do in our show the sync didn't even get close, dropping many phonemes and just basically looking like a hack job. Perhaps others don't have as much dialog or lower expectations).

Yeah, "Little Shop" was the one they remade as "You Got Mail". I do like the Jimmy Stewart version, and can't think of the actress without using IMBD either. Maureen Sullivan? Sullivan sticks in my mind for some reason. But, yeah, it's great work on the sync, and great work on the animation. As always, I admire and envy your artistic talent.

[heyvern]

Another great idea for a tutorial! Keep them coming Mike!

http://www.lowrestv.com/anime_studio/as ... -fritz.mov

First off the new lip sync feature in AS 6 is great. A huge time saver. Here's why; When you type in the text it actually gets those switch keys in the right place in the audio. No it isn't perfect on the "first go" but it's so dang close it saves a lot of work.

I did the above in about 10-15 minutes. The changes I had to make involved the "rest" switch layer. It would put those in but only at the end of a phrase so the mouth would "interpolate" slowly to the next shape for the next phrase.

I then had to edit some keys at the end because I spelled "cigerette" and "ligter" incorrectly (I did it here as well )

This could use a lot more work to be much better. But we both agree that lip sync is not "easy" and no automation can make it an exact science with out some hand editing.

This will definitely be a great tutorial.

-vern

[mkelley]

I'll let your .MOV speak for itself (and those playing along at home, just look at the first one he did with PG and the girl and compare it to this one done automatically).

It was Margaret Sullivan (I *knew* it was a Sullivan :>)

[mkelley]

Hmmm -- here's a thought (I'm throwing it out to you, Vern, because you're a great scripter, not with any hidden agenda).

Write a script to take the excellent phoneme track produced by PG and match it up to the audio done by the auto sync. IOW, just type the text in PG (make no attempt to line it up) and have PG generate the phonemes right (which it does) and then use the auto sync in AS 6 to match it up with the audio.

(This kind of begs the question on why PG does such an excellent job at generating the correct phonemes -- Mike wrote both so you'd think it would be equally good. Unless PG is calling some PD generating function that he can't use inside of AS. Would love to know the answer to this one).

It might actually work -- heck, I may give that a go myself.

[heyvern]

The "auto" lip sync in AS 6 is not supposed to be "magic". It merely states:

"For more accurate lip syncing, enter the spoken text below:

Can you explain your "challenge" in more detail? I checked over the document I did using the text I typed and the audio file and the correct phonemes were used in most cases properly and in the right spots. However in some frames they were wrong for what could be any number of reasons; the words were spoken to quickly and the auto syncing didn't get them in the right spot. I spelled the words wrong.

I often use "made up words" in papagayo. I make up strange phonetic words to force the correct "phonemes" or change how the words are formed. I suppose the same trick can be used in AS 6.

I did this originally in Papagayo some years ago. I spent 3 times as much time tweaking the phonemes in papagayo. As I said I spent all of 10 to 15 minutes in the above test. It was not intended to be PERFECT or even good because I ALREADY HAVE THE DAT FILE from years ago. I was trying to show what the results were in AS 6 with the least amount of effort. I am just demoing this.

Keep using papagayo. If it works better that's great.

I HAVE to use papagayo because the head rigs I am using now don't use switch layer lip sync. I need a "dat" file so I can import it using a script to use phoneme bone ACTIONS instead of switch layers. I then go back and tweak it and add additional emotion and adjust problem areas on the bone keys. This is how those "2.5d" head rig samples on my youtube channel were done... papagayo and actions.

-vern

[mkelley]

Can you explain your "challenge" in more detail? I checked over the document I did using the text I typed and the audio file and the correct phonemes were used in most cases properly and in the right spots. However in some frames they were wrong for what could be any number of reasons; the words were spoken to quickly and the auto syncing didn't get them in the right spot. I spelled the words wrong.

What the heck are you talking about? What or where do you see anything I wrote that eludes to a "challenge" in any way shape or form? See? It's things like this that make people think you DO have a hidden agenda or are reading things that aren't there.

The only thing I noted was for others to compare your own work with both PG and with the auto sync (that's hardly a challenge -- it's just a comparison). And I said it would speak for itself and I stand by that.

What I DID say was this -- I've tried the auto sync on the stuff we use in our show and it's not even close. It kind of reminds me of the old days of OCR -- they used to tout a "95% success rate" but the other 5% took so much time and effort to correct you might as well have typed in the stuff by hand (OCR has gotten a lot better). If I had to correct all the errors the auto sync did (without a word track to help -- reading a phoneme switch layer track is about like the old days of pen and pencil) -- I might as well have done it in PG.

I think some hobbyists will find it useful and I'm certainly not sorry it's in there for them, but I can't think of ANYONE who uses AS seriously who would use this for anything. Sorry, that's IMHO and it's not going to change.

[knunk]

Wow!
Vern bashing season continues.

MKelly you are a hobbyist right?