Lip Sync... To Interpolate Or Not To Interpolate?

Have you come up with a good Moho trick? Need help solving an animation problem? Come on in.

Moderators: Fahim, Distinct Sun, Víctor Paredes, erey, Belgarath, slowtiger

Lip Sync... To Interpolate Or Not To Interpolate?

Postby drumlug13 » Mon Sep 04, 2017 5:29 pm

Do you prefer one method over the other? Which and why?
User avatar
drumlug13
 
Posts: 457
Joined: Sun May 22, 2011 9:16 pm
Location: Beautiful St. Louis, MO

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby Greenlaw » Tue Sep 05, 2017 5:38 am

I'm assuming you mean with Moho Switch layers? I don't bother with interpolation for lip sync, in personal stuff or work. I think it makes the mouth animation look kinda a 'mushy'. What we typically do is first animate with static mouth shapes and get the timing and shapes locked in. Then we do a second pass where we the Layer Transform tool to 'squash and stretch' the Switch group itself to create in-between to help smooth out the transitions. This can make the lip-sync look surprisingly smooth with only a little effort.

Other tricks: To keep the number of drawings lower, I often add an 'emotion' control so I can move the corners of the mouth up or down using the same mouth shapes. Also, during animation, I may tweak a mouth shape using Magnet or Transform Points to match what I see in the animatic more closely. This is good for quick one offs where you don't feel like updating the master rig. If the shape is something you expect to use often though, you should add it as a new mouth drawing in the Master rig and update yor refence rig.

The above system works well for most dialog for cartoony or 'realistic' characters, and doesn't take a lot of time to do.

Of course, this may depend on just how realistic you need the lips-ync to be, or if your character is speaking very slowly. Then you may need to set up something more 'morphy' to look natural.

Note to developers: I would like to use Pin bones in the face and mouth but I wish the display size of the pin bones could be scaled independently of Scale and Strength. The default display size of pin bones is often way too big too big or way too small to use in detailed areas like around a character's mouth. The user should be allowed to adjust the Pin bone display size to fit the character as needed.

Most of the time, I wind up using regular bones instead just because I can scale them to exactly the size I need. I would prefer to use the Pin bones because of their shape and 'handling' but the current fixed display size usually kills this for me.
User avatar
Greenlaw
 
Posts: 3451
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby chucky » Tue Sep 05, 2017 5:49 am

Well I actually like the interpolation, you do have to hold phonemes not to get to much mushiness ( which can look a little chewy) but it can work awesomely , (especially if you lower the frame/interpolation rate, higher frame rates look weird in a lot of animation situations).
If you do it that way then , it just gives a little more variety and animation.
Unfortunately it doesn't work when they are inside layers for masking which is unfortunate...
Although now I'm thinking 'Hayasidist new viswitch script might help out with that...
Hmmm maybe, thanks for bringing it up Adam
... Oh hey, how's that Noir piece going? It was looking great last I saw. :!:
Sequential offender.
my latest animations
User avatar
chucky
 
Posts: 3708
Joined: Sun Jan 28, 2007 4:24 am
Location: Van Diemen's Land

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby chucky » Tue Sep 05, 2017 6:17 am

Note to developers: I would like to use Pin bones in the face and mouth but I wish the display size of the pin bones could be scaled independently of Scale and Strength. The default display size of pin bones is often way too big too big or way too small to use in detailed areas like around a character's mouth. The user should be allowed to adjust the Pin bone display size to fit the character as needed.


I agree, I'm not sure I might have reported that, I'm sure that'll get a few +1's
Sequential offender.
my latest animations
User avatar
chucky
 
Posts: 3708
Joined: Sun Jan 28, 2007 4:24 am
Location: Van Diemen's Land

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby hayasidist » Tue Sep 05, 2017 12:52 pm

chucky wrote:Unfortunately it doesn't work when they are inside layers for masking which is unfortunate... Although now I'm thinking 'Hayasidist new viswitch script might help out with that...


the master layer just needs names -- you could have (e.g.) an empty vector in the manually controlled master and a group / bone in the "auto" phonemes - there's no checking to see if the layer types match (but as you know - which is why we are where we are - there's no "exclude strokes" feature in masking in group types)

-- now all of a sudden a thought hits me: what if we could have an "exclude strokes" (i.e. show fills) option and an "exclude fills" (i.e. show strokes) option on a vector layer? that alongside copy by ref might mean we could put fills and strokes in separate groups for masking.
User avatar
hayasidist
 
Posts: 1849
Joined: Wed Feb 16, 2011 8:12 pm
Location: Kent, England

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby chucky » Tue Sep 05, 2017 1:40 pm

Well you can already do that really.
Make a ref of the group and toggle the stroke or fill, shape by shape, the problem would be overlapping shapes will not give the right pie.
Sequential offender.
my latest animations
User avatar
chucky
 
Posts: 3708
Joined: Sun Jan 28, 2007 4:24 am
Location: Van Diemen's Land

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby chucky » Tue Sep 05, 2017 2:30 pm

Would it be possible to make the script work with a bone parent as well as a switch?
Sequential offender.
my latest animations
User avatar
chucky
 
Posts: 3708
Joined: Sun Jan 28, 2007 4:24 am
Location: Van Diemen's Land

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby Greenlaw » Tue Sep 05, 2017 4:19 pm

chucky wrote:Unfortunately it doesn't work when they are inside layers for masking which is unfortunate...

Oh, yeah, I guess I should have mentioned that. Because of the limitation with nested groups in Switch Layers, most of the time it's not technically possible to use interpolation in our mouth setups anyway. :)
Last edited by Greenlaw on Tue Sep 05, 2017 5:19 pm, edited 1 time in total.
User avatar
Greenlaw
 
Posts: 3451
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby hayasidist » Tue Sep 05, 2017 4:50 pm

chucky wrote:Would it be possible to make the script work with a bone parent as well as a switch?
yes -- a couple of options here: The most straightforward is:

use groups instead of switch children - that would need a "controlling" group to be identified explicitly (??) or maybe better, implicitly -- e.g. the active group

and that would mean that the tool will need to be invoked on the controlling group (as distinct from the parent of that group) i.e. different from the way the switch version is currently invoked.

and so for consistency, it would make sense to change the switch version to work on the selected child (and not the switch itself).

IOW

Code: Select all
"daddy of them all" Group (any group type - switch / bone / group ...)
>any group type - "Item A"
>>stuff
>>type 1 option
>>type 2 option
>> ..

>any group type - "item B"
>>stuff
>>type 1 option
>>type 2 option
>> ..

...
>any group type - "item N" <<< Note no "stuff" <<<<<
>>type 1 option
>>type 2 option
>> ..

...

>any group type - "item X"
>>stuff
>>type 1 option
>>type 2 option
>> ..

Way of working:
Set vis on types 1 .. n in group "item N"
Select group N
Run tool -- and that does:
>for all groups in "daddy of them all" Group
>> set vis of types 1...n as per that set in group N
>end

By analogy of the dummy phoneme - I'd make the controlling group a "do not render" group that only has the required optional types in it - but there's no actual need to do that.

not hard conceptually. I can take a longer look at that mid of next week ....
User avatar
hayasidist
 
Posts: 1849
Joined: Wed Feb 16, 2011 8:12 pm
Location: Kent, England

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby chucky » Tue Sep 05, 2017 5:48 pm

Cool!
So that would be really flexible and powerful, especially if it didn't need a dummy group.
It could be used as a tool in files not even specifically set up for it.
Sequential offender.
my latest animations
User avatar
chucky
 
Posts: 3708
Joined: Sun Jan 28, 2007 4:24 am
Location: Van Diemen's Land

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby drumlug13 » Wed Sep 06, 2017 9:37 pm

chucky wrote:... Oh hey, how's that Noir piece going? It was looking great last I saw. :!:


Not a whole lot of progress since we last talked about it. There was a bit of burn out and a lot of Moho crashes, but mostly I stopped because I couldn't make up my mind on the character design. I had to lay off of it for a while but I'm starting to mess around with it again. I feel a little refreshed looking at it now.

As far as the lip sync goes.... I usually have everything set up for interpolation but it always end up using a lot of holds. I was starting to wonder if I should just scrap the interpolate method.
User avatar
drumlug13
 
Posts: 457
Joined: Sun May 22, 2011 9:16 pm
Location: Beautiful St. Louis, MO

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby hayasidist » Thu Sep 07, 2017 1:25 pm

chucky wrote:Would it be possible to make the script work with a bone parent as well as a switch...
So that would be really flexible and powerful, especially if it didn't need a dummy group.
It could be used as a tool in files not even specifically set up for it.


done. I found a couple of hours to do it (as ever the bundling for release takes nearly as long as the code change!) Still in the same place:
http://www.mediafire.com/file/uxu0vb959 ... Switch.zip

this is now V2.00

activate on any group type in any parent structure -- the visibility of that group's children is propagated to children of the same name in all groups in parent
User avatar
hayasidist
 
Posts: 1849
Joined: Wed Feb 16, 2011 8:12 pm
Location: Kent, England

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby chucky » Thu Sep 07, 2017 3:24 pm

Awesome Paul!
Installed and in use.
Thanks so much.
Sequential offender.
my latest animations
User avatar
chucky
 
Posts: 3708
Joined: Sun Jan 28, 2007 4:24 am
Location: Van Diemen's Land

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby niroshB » Sat Mar 17, 2018 6:31 am

chucky wrote:Awesome Paul!
Installed and in use.
Thanks so much.


Hi,

Please tell me how to use it.
thanks. :D
niroshB
 
Posts: 3
Joined: Mon Nov 07, 2016 9:15 pm

Re: Lip Sync... To Interpolate Or Not To Interpolate?

Postby Greenlaw » Sun Mar 18, 2018 2:25 am

As an experiment, a few weeks ago I set up a character at work using the new group interpolation feature in 12.4 and this worked amazingly well. The setup complicated the animation process a little but the result looked very fluid and natural.

We’re still not bothering to do this routinely because we’re not scheduled for the extra time required for setup and animation but this feature definitely opens new possibilities for future shows.

FYI, I couldn’t use my normal SBD setup for this, and used Switch Selection instead. This worked out ok but Switch Selection sill has the problems the artwork displaying transformed by bones and not staying ‘in place’, which gets annoying as heck to lipsync with when you just want to view at the original untransformed artwork.

Switch Selection really need to keep the drawings static like the Drawing Substitution window in Toon Boom, or at least have this option available. This is the only reason why we still avoid using Switch Selection in Moho...it would otherwise be an incredibly useful feature, but at the moment it’s a very annoying one.

Anyway, I can’t show what I did at work of course becsause the production isn’t out yet but maybe I can find time to create a generic example.
Last edited by Greenlaw on Sun Mar 18, 2018 2:47 am, edited 1 time in total.
User avatar
Greenlaw
 
Posts: 3451
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles

Next

Return to Tips & Techniques

Who is online

Users browsing this forum: ResvStar and 1 guest