Lip Sync... To Interpolate Or Not To Interpolate?
Moderators: Víctor Paredes, Belgarath, slowtiger
Lip Sync... To Interpolate Or Not To Interpolate?
Do you prefer one method over the other? Which and why?
Adam
Re: Lip Sync... To Interpolate Or Not To Interpolate?
I'm assuming you mean with Moho Switch layers? I don't bother with interpolation for lip sync, in personal stuff or work. I think it makes the mouth animation look kinda a 'mushy'. What we typically do is first animate with static mouth shapes and get the timing and shapes locked in. Then we do a second pass where we the Layer Transform tool to 'squash and stretch' the Switch group itself to create in-between to help smooth out the transitions. This can make the lip-sync look surprisingly smooth with only a little effort.
Other tricks: To keep the number of drawings lower, I often add an 'emotion' control so I can move the corners of the mouth up or down using the same mouth shapes. Also, during animation, I may tweak a mouth shape using Magnet or Transform Points to match what I see in the animatic more closely. This is good for quick one offs where you don't feel like updating the master rig. If the shape is something you expect to use often though, you should add it as a new mouth drawing in the Master rig and update yor refence rig.
The above system works well for most dialog for cartoony or 'realistic' characters, and doesn't take a lot of time to do.
Of course, this may depend on just how realistic you need the lips-ync to be, or if your character is speaking very slowly. Then you may need to set up something more 'morphy' to look natural.
Note to developers: I would like to use Pin bones in the face and mouth but I wish the display size of the pin bones could be scaled independently of Scale and Strength. The default display size of pin bones is often way too big too big or way too small to use in detailed areas like around a character's mouth. The user should be allowed to adjust the Pin bone display size to fit the character as needed.
Most of the time, I wind up using regular bones instead just because I can scale them to exactly the size I need. I would prefer to use the Pin bones because of their shape and 'handling' but the current fixed display size usually kills this for me.
Other tricks: To keep the number of drawings lower, I often add an 'emotion' control so I can move the corners of the mouth up or down using the same mouth shapes. Also, during animation, I may tweak a mouth shape using Magnet or Transform Points to match what I see in the animatic more closely. This is good for quick one offs where you don't feel like updating the master rig. If the shape is something you expect to use often though, you should add it as a new mouth drawing in the Master rig and update yor refence rig.
The above system works well for most dialog for cartoony or 'realistic' characters, and doesn't take a lot of time to do.
Of course, this may depend on just how realistic you need the lips-ync to be, or if your character is speaking very slowly. Then you may need to set up something more 'morphy' to look natural.
Note to developers: I would like to use Pin bones in the face and mouth but I wish the display size of the pin bones could be scaled independently of Scale and Strength. The default display size of pin bones is often way too big too big or way too small to use in detailed areas like around a character's mouth. The user should be allowed to adjust the Pin bone display size to fit the character as needed.
Most of the time, I wind up using regular bones instead just because I can scale them to exactly the size I need. I would prefer to use the Pin bones because of their shape and 'handling' but the current fixed display size usually kills this for me.
D.R. Greenlaw
Artist/Partner - Little Green Dog | Little Green Dog Channel on Vimeo | Greenlaw's Demo Reel 2020 Edtion
Artist/Partner - Little Green Dog | Little Green Dog Channel on Vimeo | Greenlaw's Demo Reel 2020 Edtion
Re: Lip Sync... To Interpolate Or Not To Interpolate?
Well I actually like the interpolation, you do have to hold phonemes not to get to much mushiness ( which can look a little chewy) but it can work awesomely , (especially if you lower the frame/interpolation rate, higher frame rates look weird in a lot of animation situations).
If you do it that way then , it just gives a little more variety and animation.
Unfortunately it doesn't work when they are inside layers for masking which is unfortunate...
Although now I'm thinking 'Hayasidist new viswitch script might help out with that...
Hmmm maybe, thanks for bringing it up Adam
... Oh hey, how's that Noir piece going? It was looking great last I saw.
If you do it that way then , it just gives a little more variety and animation.
Unfortunately it doesn't work when they are inside layers for masking which is unfortunate...
Although now I'm thinking 'Hayasidist new viswitch script might help out with that...
Hmmm maybe, thanks for bringing it up Adam
... Oh hey, how's that Noir piece going? It was looking great last I saw.
Re: Lip Sync... To Interpolate Or Not To Interpolate?
I agree, I'm not sure I might have reported that, I'm sure that'll get a few +1'sNote to developers: I would like to use Pin bones in the face and mouth but I wish the display size of the pin bones could be scaled independently of Scale and Strength. The default display size of pin bones is often way too big too big or way too small to use in detailed areas like around a character's mouth. The user should be allowed to adjust the Pin bone display size to fit the character as needed.
- hayasidist
- Posts: 3573
- Joined: Wed Feb 16, 2011 8:12 pm
- Location: Kent, England
Re: Lip Sync... To Interpolate Or Not To Interpolate?
the master layer just needs names -- you could have (e.g.) an empty vector in the manually controlled master and a group / bone in the "auto" phonemes - there's no checking to see if the layer types match (but as you know - which is why we are where we are - there's no "exclude strokes" feature in masking in group types)chucky wrote:Unfortunately it doesn't work when they are inside layers for masking which is unfortunate... Although now I'm thinking 'Hayasidist new viswitch script might help out with that...
-- now all of a sudden a thought hits me: what if we could have an "exclude strokes" (i.e. show fills) option and an "exclude fills" (i.e. show strokes) option on a vector layer? that alongside copy by ref might mean we could put fills and strokes in separate groups for masking.
Re: Lip Sync... To Interpolate Or Not To Interpolate?
Well you can already do that really.
Make a ref of the group and toggle the stroke or fill, shape by shape, the problem would be overlapping shapes will not give the right pie.
Make a ref of the group and toggle the stroke or fill, shape by shape, the problem would be overlapping shapes will not give the right pie.
Re: Lip Sync... To Interpolate Or Not To Interpolate?
Would it be possible to make the script work with a bone parent as well as a switch?
Re: Lip Sync... To Interpolate Or Not To Interpolate?
Oh, yeah, I guess I should have mentioned that. Because of the limitation with nested groups in Switch Layers, most of the time it's not technically possible to use interpolation in our mouth setups anyway.chucky wrote:Unfortunately it doesn't work when they are inside layers for masking which is unfortunate...
Last edited by Greenlaw on Tue Sep 05, 2017 5:19 pm, edited 1 time in total.
D.R. Greenlaw
Artist/Partner - Little Green Dog | Little Green Dog Channel on Vimeo | Greenlaw's Demo Reel 2020 Edtion
Artist/Partner - Little Green Dog | Little Green Dog Channel on Vimeo | Greenlaw's Demo Reel 2020 Edtion
- hayasidist
- Posts: 3573
- Joined: Wed Feb 16, 2011 8:12 pm
- Location: Kent, England
Re: Lip Sync... To Interpolate Or Not To Interpolate?
yes -- a couple of options here: The most straightforward is:chucky wrote:Would it be possible to make the script work with a bone parent as well as a switch?
use groups instead of switch children - that would need a "controlling" group to be identified explicitly (??) or maybe better, implicitly -- e.g. the active group
and that would mean that the tool will need to be invoked on the controlling group (as distinct from the parent of that group) i.e. different from the way the switch version is currently invoked.
and so for consistency, it would make sense to change the switch version to work on the selected child (and not the switch itself).
IOW
Code: Select all
"daddy of them all" Group (any group type - switch / bone / group ...)
>any group type - "Item A"
>>stuff
>>type 1 option
>>type 2 option
>> ..
>any group type - "item B"
>>stuff
>>type 1 option
>>type 2 option
>> ..
...
>any group type - "item N" <<< Note no "stuff" <<<<<
>>type 1 option
>>type 2 option
>> ..
...
>any group type - "item X"
>>stuff
>>type 1 option
>>type 2 option
>> ..
Set vis on types 1 .. n in group "item N"
Select group N
Run tool -- and that does:
>for all groups in "daddy of them all" Group
>> set vis of types 1...n as per that set in group N
>end
By analogy of the dummy phoneme - I'd make the controlling group a "do not render" group that only has the required optional types in it - but there's no actual need to do that.
not hard conceptually. I can take a longer look at that mid of next week ....
Re: Lip Sync... To Interpolate Or Not To Interpolate?
Cool!
So that would be really flexible and powerful, especially if it didn't need a dummy group.
It could be used as a tool in files not even specifically set up for it.
So that would be really flexible and powerful, especially if it didn't need a dummy group.
It could be used as a tool in files not even specifically set up for it.
Re: Lip Sync... To Interpolate Or Not To Interpolate?
Not a whole lot of progress since we last talked about it. There was a bit of burn out and a lot of Moho crashes, but mostly I stopped because I couldn't make up my mind on the character design. I had to lay off of it for a while but I'm starting to mess around with it again. I feel a little refreshed looking at it now.chucky wrote:... Oh hey, how's that Noir piece going? It was looking great last I saw.
As far as the lip sync goes.... I usually have everything set up for interpolation but it always end up using a lot of holds. I was starting to wonder if I should just scrap the interpolate method.
Adam
- hayasidist
- Posts: 3573
- Joined: Wed Feb 16, 2011 8:12 pm
- Location: Kent, England
Re: Lip Sync... To Interpolate Or Not To Interpolate?
done. I found a couple of hours to do it (as ever the bundling for release takes nearly as long as the code change!) Still in the same place:chucky wrote:Would it be possible to make the script work with a bone parent as well as a switch...
So that would be really flexible and powerful, especially if it didn't need a dummy group.
It could be used as a tool in files not even specifically set up for it.
http://www.mediafire.com/file/uxu0vb959 ... Switch.zip
this is now V2.00
activate on any group type in any parent structure -- the visibility of that group's children is propagated to children of the same name in all groups in parent
Re: Lip Sync... To Interpolate Or Not To Interpolate?
Awesome Paul!
Installed and in use.
Thanks so much.
Installed and in use.
Thanks so much.
Re: Lip Sync... To Interpolate Or Not To Interpolate?
Hi,chucky wrote:Awesome Paul!
Installed and in use.
Thanks so much.
Please tell me how to use it.
thanks.
Re: Lip Sync... To Interpolate Or Not To Interpolate?
As an experiment, a few weeks ago I set up a character at work using the new group interpolation feature in 12.4 and this worked amazingly well. The setup complicated the animation process a little but the result looked very fluid and natural.
We’re still not bothering to do this routinely because we’re not scheduled for the extra time required for setup and animation but this feature definitely opens new possibilities for future shows.
FYI, I couldn’t use my normal SBD setup for this, and used Switch Selection instead. This worked out ok but Switch Selection sill has the problems the artwork displaying transformed by bones and not staying ‘in place’, which gets annoying as heck to lipsync with when you just want to view at the original untransformed artwork.
Switch Selection really need to keep the drawings static like the Drawing Substitution window in Toon Boom, or at least have this option available. This is the only reason why we still avoid using Switch Selection in Moho...it would otherwise be an incredibly useful feature, but at the moment it’s a very annoying one.
Anyway, I can’t show what I did at work of course becsause the production isn’t out yet but maybe I can find time to create a generic example.
We’re still not bothering to do this routinely because we’re not scheduled for the extra time required for setup and animation but this feature definitely opens new possibilities for future shows.
FYI, I couldn’t use my normal SBD setup for this, and used Switch Selection instead. This worked out ok but Switch Selection sill has the problems the artwork displaying transformed by bones and not staying ‘in place’, which gets annoying as heck to lipsync with when you just want to view at the original untransformed artwork.
Switch Selection really need to keep the drawings static like the Drawing Substitution window in Toon Boom, or at least have this option available. This is the only reason why we still avoid using Switch Selection in Moho...it would otherwise be an incredibly useful feature, but at the moment it’s a very annoying one.
Anyway, I can’t show what I did at work of course becsause the production isn’t out yet but maybe I can find time to create a generic example.
Last edited by Greenlaw on Sun Mar 18, 2018 2:47 am, edited 1 time in total.
D.R. Greenlaw
Artist/Partner - Little Green Dog | Little Green Dog Channel on Vimeo | Greenlaw's Demo Reel 2020 Edtion
Artist/Partner - Little Green Dog | Little Green Dog Channel on Vimeo | Greenlaw's Demo Reel 2020 Edtion