Lip Sync... To Interpolate Or Not To Interpolate?

Have you come up with a good Moho trick? Need help solving an animation problem? Come on in.

Moderators: Víctor Paredes, Belgarath, slowtiger

User avatar
drumlug13
Posts: 537
Joined: Sun May 22, 2011 9:16 pm
Location: Beautiful St. Louis, MO

Lip Sync... To Interpolate Or Not To Interpolate?

Post by drumlug13 »

Do you prefer one method over the other? Which and why?
Adam
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by Greenlaw »

I'm assuming you mean with Moho Switch layers? I don't bother with interpolation for lip sync, in personal stuff or work. I think it makes the mouth animation look kinda a 'mushy'. What we typically do is first animate with static mouth shapes and get the timing and shapes locked in. Then we do a second pass where we the Layer Transform tool to 'squash and stretch' the Switch group itself to create in-between to help smooth out the transitions. This can make the lip-sync look surprisingly smooth with only a little effort.

Other tricks: To keep the number of drawings lower, I often add an 'emotion' control so I can move the corners of the mouth up or down using the same mouth shapes. Also, during animation, I may tweak a mouth shape using Magnet or Transform Points to match what I see in the animatic more closely. This is good for quick one offs where you don't feel like updating the master rig. If the shape is something you expect to use often though, you should add it as a new mouth drawing in the Master rig and update yor refence rig.

The above system works well for most dialog for cartoony or 'realistic' characters, and doesn't take a lot of time to do.

Of course, this may depend on just how realistic you need the lips-ync to be, or if your character is speaking very slowly. Then you may need to set up something more 'morphy' to look natural.

Note to developers: I would like to use Pin bones in the face and mouth but I wish the display size of the pin bones could be scaled independently of Scale and Strength. The default display size of pin bones is often way too big too big or way too small to use in detailed areas like around a character's mouth. The user should be allowed to adjust the Pin bone display size to fit the character as needed.

Most of the time, I wind up using regular bones instead just because I can scale them to exactly the size I need. I would prefer to use the Pin bones because of their shape and 'handling' but the current fixed display size usually kills this for me.
chucky
Posts: 4650
Joined: Sun Jan 28, 2007 4:24 am

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by chucky »

Well I actually like the interpolation, you do have to hold phonemes not to get to much mushiness ( which can look a little chewy) but it can work awesomely , (especially if you lower the frame/interpolation rate, higher frame rates look weird in a lot of animation situations).
If you do it that way then , it just gives a little more variety and animation.
Unfortunately it doesn't work when they are inside layers for masking which is unfortunate...
Although now I'm thinking 'Hayasidist new viswitch script might help out with that...
Hmmm maybe, thanks for bringing it up Adam
... Oh hey, how's that Noir piece going? It was looking great last I saw. :!:
chucky
Posts: 4650
Joined: Sun Jan 28, 2007 4:24 am

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by chucky »

Note to developers: I would like to use Pin bones in the face and mouth but I wish the display size of the pin bones could be scaled independently of Scale and Strength. The default display size of pin bones is often way too big too big or way too small to use in detailed areas like around a character's mouth. The user should be allowed to adjust the Pin bone display size to fit the character as needed.
I agree, I'm not sure I might have reported that, I'm sure that'll get a few +1's
User avatar
hayasidist
Posts: 3492
Joined: Wed Feb 16, 2011 8:12 pm
Location: Kent, England

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by hayasidist »

chucky wrote:Unfortunately it doesn't work when they are inside layers for masking which is unfortunate... Although now I'm thinking 'Hayasidist new viswitch script might help out with that...
the master layer just needs names -- you could have (e.g.) an empty vector in the manually controlled master and a group / bone in the "auto" phonemes - there's no checking to see if the layer types match (but as you know - which is why we are where we are - there's no "exclude strokes" feature in masking in group types)

-- now all of a sudden a thought hits me: what if we could have an "exclude strokes" (i.e. show fills) option and an "exclude fills" (i.e. show strokes) option on a vector layer? that alongside copy by ref might mean we could put fills and strokes in separate groups for masking.
chucky
Posts: 4650
Joined: Sun Jan 28, 2007 4:24 am

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by chucky »

Well you can already do that really.
Make a ref of the group and toggle the stroke or fill, shape by shape, the problem would be overlapping shapes will not give the right pie.
chucky
Posts: 4650
Joined: Sun Jan 28, 2007 4:24 am

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by chucky »

Would it be possible to make the script work with a bone parent as well as a switch?
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by Greenlaw »

chucky wrote:Unfortunately it doesn't work when they are inside layers for masking which is unfortunate...
Oh, yeah, I guess I should have mentioned that. Because of the limitation with nested groups in Switch Layers, most of the time it's not technically possible to use interpolation in our mouth setups anyway. :)
Last edited by Greenlaw on Tue Sep 05, 2017 5:19 pm, edited 1 time in total.
User avatar
hayasidist
Posts: 3492
Joined: Wed Feb 16, 2011 8:12 pm
Location: Kent, England

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by hayasidist »

chucky wrote:Would it be possible to make the script work with a bone parent as well as a switch?
yes -- a couple of options here: The most straightforward is:

use groups instead of switch children - that would need a "controlling" group to be identified explicitly (??) or maybe better, implicitly -- e.g. the active group

and that would mean that the tool will need to be invoked on the controlling group (as distinct from the parent of that group) i.e. different from the way the switch version is currently invoked.

and so for consistency, it would make sense to change the switch version to work on the selected child (and not the switch itself).

IOW

Code: Select all

"daddy of them all" Group (any group type - switch / bone / group ...)
>any group type - "Item A"
>>stuff
>>type 1 option
>>type 2 option
>> ..

>any group type - "item B"
>>stuff
>>type 1 option
>>type 2 option
>> ..

...
>any group type - "item N" <<< Note no "stuff" <<<<<
>>type 1 option
>>type 2 option
>> ..

...

>any group type - "item X"
>>stuff
>>type 1 option
>>type 2 option
>> ..
Way of working:
Set vis on types 1 .. n in group "item N"
Select group N
Run tool -- and that does:
>for all groups in "daddy of them all" Group
>> set vis of types 1...n as per that set in group N
>end

By analogy of the dummy phoneme - I'd make the controlling group a "do not render" group that only has the required optional types in it - but there's no actual need to do that.

not hard conceptually. I can take a longer look at that mid of next week ....
chucky
Posts: 4650
Joined: Sun Jan 28, 2007 4:24 am

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by chucky »

Cool!
So that would be really flexible and powerful, especially if it didn't need a dummy group.
It could be used as a tool in files not even specifically set up for it.
User avatar
drumlug13
Posts: 537
Joined: Sun May 22, 2011 9:16 pm
Location: Beautiful St. Louis, MO

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by drumlug13 »

chucky wrote:... Oh hey, how's that Noir piece going? It was looking great last I saw. :!:
Not a whole lot of progress since we last talked about it. There was a bit of burn out and a lot of Moho crashes, but mostly I stopped because I couldn't make up my mind on the character design. I had to lay off of it for a while but I'm starting to mess around with it again. I feel a little refreshed looking at it now.

As far as the lip sync goes.... I usually have everything set up for interpolation but it always end up using a lot of holds. I was starting to wonder if I should just scrap the interpolate method.
Adam
User avatar
hayasidist
Posts: 3492
Joined: Wed Feb 16, 2011 8:12 pm
Location: Kent, England

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by hayasidist »

chucky wrote:Would it be possible to make the script work with a bone parent as well as a switch...
So that would be really flexible and powerful, especially if it didn't need a dummy group.
It could be used as a tool in files not even specifically set up for it.
done. I found a couple of hours to do it (as ever the bundling for release takes nearly as long as the code change!) Still in the same place:
http://www.mediafire.com/file/uxu0vb959 ... Switch.zip

this is now V2.00

activate on any group type in any parent structure -- the visibility of that group's children is propagated to children of the same name in all groups in parent
chucky
Posts: 4650
Joined: Sun Jan 28, 2007 4:24 am

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by chucky »

Awesome Paul!
Installed and in use.
Thanks so much.
niroshB
Posts: 3
Joined: Mon Nov 07, 2016 9:15 pm

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by niroshB »

chucky wrote:Awesome Paul!
Installed and in use.
Thanks so much.
Hi,

Please tell me how to use it.
thanks. :D
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Post by Greenlaw »

As an experiment, a few weeks ago I set up a character at work using the new group interpolation feature in 12.4 and this worked amazingly well. The setup complicated the animation process a little but the result looked very fluid and natural.

We’re still not bothering to do this routinely because we’re not scheduled for the extra time required for setup and animation but this feature definitely opens new possibilities for future shows.

FYI, I couldn’t use my normal SBD setup for this, and used Switch Selection instead. This worked out ok but Switch Selection sill has the problems the artwork displaying transformed by bones and not staying ‘in place’, which gets annoying as heck to lipsync with when you just want to view at the original untransformed artwork.

Switch Selection really need to keep the drawings static like the Drawing Substitution window in Toon Boom, or at least have this option available. This is the only reason why we still avoid using Switch Selection in Moho...it would otherwise be an incredibly useful feature, but at the moment it’s a very annoying one.

Anyway, I can’t show what I did at work of course becsause the production isn’t out yet but maybe I can find time to create a generic example.
Last edited by Greenlaw on Sun Mar 18, 2018 2:47 am, edited 1 time in total.
Post Reply