Need Help Syncing Mouth with Audio

General Moho topics.

Moderators: Víctor Paredes, Belgarath, slowtiger

chiefr_suthrland
Posts: 55
Joined: Tue Apr 11, 2017 9:25 pm
Location: Los Angeles, CA

Need Help Syncing Mouth with Audio

Post by chiefr_suthrland »

Quick question, I'm trying to sync my character's lips with the audio and it seemed easy enough...until I played it back from "Frame 1". As the "mouth" smart bone dials, every mouth shape passed appears on my character's face. So, for example, if I have 16 different mouth shapes assigned to 16 different frames on the "mouth smart bone and I'm trying to mouth the word "shackle". Let's say the "Sh" mouth shape is on "Frame 1" (in smart bone mode with "mouth" selected) & the "ah" mouth shape is on "Frame 16"...I'll go to the general timeline and put "Sh" on "Frame 1" and put "Ah" on "Frame 3", but on "Frame 2" I'll see frames 2-15 ("mouth" smart bone frames) appear speedily. Get what I'm trying to say? I know it sounds a bit confusing.
Last edited by chiefr_suthrland on Sat Dec 23, 2017 2:06 pm, edited 1 time in total.
chucky
Posts: 4650
Joined: Sun Jan 28, 2007 4:24 am

Re: Need Help Syncing Mouth with Audio

Post by chucky »

Generally speaking, mouths smooth out and approximate the sounds they make.
If you put every letter in too accurately to the actually words it looks weird and chattery.
Also our brains expect to see the mouth move 'before' the sound comes out.
I remember when my twin was editing for Disney he said , as a rule , if the sound was ' too dead on' they/he would slip it one frame later. The little delay would make the sync more natural and the viewer would more easily find it believable.
These are just little tips that might help reduce 'weirdness'.
User avatar
synthsin75
Posts: 9934
Joined: Mon Jan 14, 2008 11:20 pm
Location: Oklahoma
Contact:

Re: Need Help Syncing Mouth with Audio

Post by synthsin75 »

If you're using a smart bone for lip sync, you'll want to set its keyframe interpolation to step, so you don't see all the intermediate phonemes.
chiefr_suthrland
Posts: 55
Joined: Tue Apr 11, 2017 9:25 pm
Location: Los Angeles, CA

Re: Need Help Syncing Mouth with Audio

Post by chiefr_suthrland »

synthsin75 wrote:If you're using a smart bone for lip sync, you'll want to set its keyframe interpolation to step, so you don't see all the intermediate phonemes.
I set it to "Step"...unfortunately, that did not resolve this issue.

So here is the one mouth formation that I actually did set in the timeline. I set it at Frame 85...that's when the dialogue begins. You probably can't tell due to the tiny font in my screenshot, but instead of starting at Frame 85...where I placed it...it actually appears at Frame 69. Why...no clue.

Image

Now, the remaining screenshots are of all the other mouth formations I created appearing as the "mouth" smart bone dials down, as I mentioned before.

The 1st one appears on Frame 14

Image

Frame 23

Image

Frame 32

Image

Frame 42

Image

Frame 54

Image

Again, I'm not sure if you can read the tiny font in my screenshots, but the ONLY frame I applied one of my mouth options to was Frame 85 and even that one didn't appear on the correct frame. Can you notice anything in the provided screenshots that might be causing this?
User avatar
synthsin75
Posts: 9934
Joined: Mon Jan 14, 2008 11:20 pm
Location: Oklahoma
Contact:

Re: Need Help Syncing Mouth with Audio

Post by synthsin75 »

You're getting smooth interpolation from frame zero.
You need to change the frame zero interpolation to step as well. Interpolation settings effect how things animate after the keyframe.
chiefr_suthrland
Posts: 55
Joined: Tue Apr 11, 2017 9:25 pm
Location: Los Angeles, CA

Re: Need Help Syncing Mouth with Audio

Post by chiefr_suthrland »

synthsin75 wrote:You're getting smooth interpolation from frame zero.
You need to change the frame zero interpolation to step as well. Interpolation settings effect how things animate after the keyframe.
But I do have Frame 0 set to "Step"

Image
User avatar
synthsin75
Posts: 9934
Joined: Mon Jan 14, 2008 11:20 pm
Location: Oklahoma
Contact:

Re: Need Help Syncing Mouth with Audio

Post by synthsin75 »

chiefr_suthrland wrote:
synthsin75 wrote:You're getting smooth interpolation from frame zero.
You need to change the frame zero interpolation to step as well. Interpolation settings effect how things animate after the keyframe.
But I do have Frame 0 set to "Step"

Image
You didn't in any of the above images, and I can't tell with this one (too small).
chiefr_suthrland
Posts: 55
Joined: Tue Apr 11, 2017 9:25 pm
Location: Los Angeles, CA

Re: Need Help Syncing Mouth with Audio

Post by chiefr_suthrland »

synthsin75 wrote:
chiefr_suthrland wrote:
synthsin75 wrote:You're getting smooth interpolation from frame zero.
You need to change the frame zero interpolation to step as well. Interpolation settings effect how things animate after the keyframe.
But I do have Frame 0 set to "Step"

Image
You didn't in any of the above images, and I can't tell with this one (too small).
Yea. My laptop's being weird. Some screenshots come out big...others come out small. But I have a feeling we're talking about 2 separate things. Because none of the initial screenshots I provided was of Frame 0...& in the screenshots that "weren't" ridiculously small you can clearly see "Step" selected at the bottom left. Are you seeing something that I'm not seeing?

Image
Last edited by chiefr_suthrland on Sun Dec 24, 2017 1:09 am, edited 1 time in total.
User avatar
synthsin75
Posts: 9934
Joined: Mon Jan 14, 2008 11:20 pm
Location: Oklahoma
Contact:

Re: Need Help Syncing Mouth with Audio

Post by synthsin75 »

chiefr_suthrland wrote:Are you seeing something that I'm not seeing?

Image
Yes, the selected interpolation doesn't change the interpolation for existing keyframes. So I can see the frame zero keyframes on your timeline are still smooth keyframes. You'll want to right-click those and change them to step.
chiefr_suthrland
Posts: 55
Joined: Tue Apr 11, 2017 9:25 pm
Location: Los Angeles, CA

Re: Need Help Syncing Mouth with Audio

Post by chiefr_suthrland »

synthsin75 wrote:Yes, the selected interpolation doesn't change the interpolation for existing keyframes. So I can see the frame zero keyframes on your timeline are still smooth keyframes. You'll want to right-click those and change them to step.
Okay. So after I changed them from "Smooth" to "Step", I ran into another problem...and another problem after that lol. I got everything working now though...finally! Synth to the rescue again. I'll probably need you or Greenlaw to rescue me again soon, so don't take your cape off just yet lol. Happy holidays, my friend! :mrgreen:
Last edited by chiefr_suthrland on Sun Dec 24, 2017 3:55 am, edited 1 time in total.
User avatar
synthsin75
Posts: 9934
Joined: Mon Jan 14, 2008 11:20 pm
Location: Oklahoma
Contact:

Re: Need Help Syncing Mouth with Audio

Post by synthsin75 »

chiefr_suthrland wrote:
synthsin75 wrote:Yes, the selected interpolation doesn't change the interpolation for existing keyframes. So I can see the frame zero keyframes on your timeline are still smooth keyframes. You'll want to right-click those and change them to step.
Okay. So after I changed them from "Smooth" to "Step", I ran into another problem...and another problem after that lol. I got everything working now though...finally! Synth to the rescue again. I'll probably need you or Greenlaw to rescue me again soon, so don't take your cape of just yet lol. Happy holidays, my friend! :mrgreen:
Glad to hear you got it sorted.
Aside from holiday celebrations, I'm sure we'll be around for the next issue.

Merry Christmas!
User avatar
Greenlaw
Posts: 9191
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: Need Help Syncing Mouth with Audio

Post by Greenlaw »

Hi,

I just saw this thread and only skimmed through it, so apologies if I repeat anything already mentioned.

Some quick tips:

In the SBD action, I make my first keyframe at frame one and set this to linear. Frame one's switch drawing should be the same as frame 0. Then the rest of the action should include all the other drawings in sequence. This way, every drawing gets equal exposure because of the linear interpolation.

Frame 0 for the SBD, as discussed, should be set to Step. This is why I make a keyframe in the Action at frame 1: it's so the Action can have a different interpolation mode (Linear) from the Mainline (Step.)

Now here's a very important tip:

I noticed you set your default interpolation mode to Step. Don't do that. Instead, set it to Copy Previous Key.

If you set this to Step, every new keyframe you create for any item will become Step unless you change it to something else...which will get annoying as heck because you're going to be constantly switching back and forth to something else endlessly.

If you set this to Copy Previous Key, then Moho will use the interpolation mode from frame 0 (or whatever the previous key is.) In this case, the switch layer SBD will only use Step but anything else you animate will use whatever mode you have set for frame 0 (usually Smooth.) This is much easier to work with because you won't need to be concerned about constantly re-setting the default mode.

BTW, Copy Previous Key essentially mimics how keyframing works in After Effects. This what you want most of the time during animation. In fact, I almost never change it to anything else. IMO, this should be the default mode for Moho after you first install it.

Hope this helps. Good luck! :)
Last edited by Greenlaw on Tue Dec 26, 2017 4:50 pm, edited 4 times in total.
User avatar
synthsin75
Posts: 9934
Joined: Mon Jan 14, 2008 11:20 pm
Location: Oklahoma
Contact:

Re: Need Help Syncing Mouth with Audio

Post by synthsin75 »

I agree. I ALWAYS leave mine set to Copy Previous Key.
chiefr_suthrland
Posts: 55
Joined: Tue Apr 11, 2017 9:25 pm
Location: Los Angeles, CA

Re: Need Help Syncing Mouth with Audio

Post by chiefr_suthrland »

Greenlaw wrote:Hi,

I just saw this thread and only skimmed through it, so apologies if I repeat anything already mentioned.

Some quick tips:

In the SBD action, I make my first keyframe at frame one and set this to linear. Frame one's switch drawing should be the same as frame 0. Then the rest of the action should include all the other drawings in sequence. This way, every drawing gets equal exposure because of the linear interpolation.

Frame 0 for the SBD, as discussed, should be set to Step. This is why I make a keyframe in the Action at frame 1: it's so the Action can have a different interpolation mode (Linear) from the Mainline (Step.)

Now here's a very important tip:

I noticed you set your default interpolation mode to Step. Don't do that. Instead, set it to Copy Previous Key.

If you set this to Step, every new keyframe you create for any item will become Step unless you change it to something else...which will get annoying as heck because you're going to be constantly switching back and forth to something else endlessly.

If you set this to Copy Previous Key, then Moho will use the interpolation mode from frame 0 (or whatever the previous key is.) In this case, the switch layer SBD will only use Step but anything else you animate will use whatever mode you have set for frame 0 (usually Smooth.) This is much easier to work with because you won't need to be concerned about constantly re-setting the default mode.

BTW, Copy Previous Key essentially mimics how keyframing works in After Effects. This what you want most of the time during animation. In fact, I almost never change it to anything else. IMO, this should be the default mode for Moho after you first install it.

Hope this helps. Good luck! :)
Thanks, man. You're definitely the most helpful dude on this site...you & Synth. I didn't set it to Step btw. Synth recommended that to fix the issue I was having...& it worked :) It actually was set to "Smooth" by default. Quick question though, what the heck does "SBD" stand for lol. Also, I don't see that "Copy Previous Key" option anywhere.
User avatar
synthsin75
Posts: 9934
Joined: Mon Jan 14, 2008 11:20 pm
Location: Oklahoma
Contact:

Re: Need Help Syncing Mouth with Audio

Post by synthsin75 »

SBD is smart bone dial...using a smart bone as a control rather than a part of a skeleton.

Copy Previous should be at the very bottom of the default interpolation list.
Post Reply