Switch layer interpolation with mesh warp image layers

[madrobot]

I'm working on a hybrid stop motion/mesh warp switch layer workflow.
While switch layer interpolation morphs vector layers between switch keys, it doesn't appear to be working on mesh warp image layers. To be clear, in lip sync with vector layer phonemes, you get nice interpolation. I want those points to drive image mesh warps, and switch image at the keys to the image for that phoneme. So the first phoneme image is distorted by mesh warp, with the points morphing towards the next phoneme (with the same number of points).
This issue reminds me of the reference fix we used to use for vector layer groups to do masking in phonemes, before we had switch layer groups. Could something similar apply in this case?

[Greenlaw]

Although I hadn't tried this before, it seemed like Mesh Warp should work with Interpolate Sublayers enabled in the Switch layer, so I threw together a quick proof to test this. This isn't meant to be a 'good' example, just a quick-and-dirty demo of the feature working...



I used the new Create SmartWarp layer command to create the mesh warp layers for each of the mouth shape drawings in the Switch, and as you can see here they do interpolate smoothly. (Well, technically speaking anyway.)

For this demo, I created four mouth shapes: Neutral, OpenWide, OpenSmall, and Smile. Phonetic mouth shapes can be added of course by duplicating and modifying the mesh warp layer.

The key, as usual, is to base each new mouth shape mesh on the same original mouth shape mesh. If you add or remove any points in a copy, that will break the interpolation. Also, how you animate the switch layer makes a difference. It works if you use any of the traditional methods for switching, which includes: the Switch Selection window, right-clicking the switch layer, Ctrl-Alt-click over the mouth, the list in the Tool Options, or the Alt-D/C shortcuts. But if you use a Smart Bone Dial, it will not work properly because you need to use Step keys to make it switch non-linearly, which by its nature doesn't allow for interpolation.

Hope this helps.

If I have time tonight, I'll do a more polished version of this setup and post it for examination. This was actually fun and really easy to do...I just want to do it well now.

[Greenlaw]

Even though the vector is animating smoothly, looking at it closely, I can see see a slight 'pop' in the image between switches. I think this issue is caused by the mouth shapes being reposed at frame 0 so the corresponding points are displacing different regions of the image when switched.

Hmm. Will have to think about how to solve that problem when I get back to this.

Edit: Ok, I think I know how to fix that. Will confirm tonight.

[madrobot]

I can't tell you how excited I am about your posts about this. Please do keep the updates coming!

[Greenlaw]

Much better...



The trick is to use the same 'baseline' mesh at 0 in every shape and keyframe the unique shapes at frame 1. Luckily, I didn't have to redo anything, I just keyframed at frame 1 for each mouth shape, and then copied frame 0 from the original mesh layer to frame 0 for all of the others.

I'm going to re-do all this anyway for the 'good' version tonight, but glad to have solved this issue already.

BTW, here's what the mesh warp layer looks like...



The mouth is literally just a square which is why this looks kinda crappy right now.

[madrobot]

So the image on each mesh warp switch is the same image? I mean, it still proves the concept to a degree as they are distinct images which happen to be identical - but what I am wanting to do is have a different image on each switch. Which is complicated because I'll draw the mesh with the same number of points, not distort the base mesh pose. Draw the target mesh shape over the photo of a sculpted phoneme. So rest phoneme morphs towards "A" phoneme, and hits the swtich key, image switches to "A" sculpt, with mesh draw open around the "A" shape (same number of points). That should work do you think?

[Greenlaw]

It sounds you want to morph between very different images, and have elements within that image morph independently. That's possible in Moho but it's not automatic, and the setup could get pretty complicated. You probably need to break out the character's head and face parts into separate switches so you can do independent morphing in each. Being able to control all this in a non-linear manner is probably where it gets really complicated.

It would help if you could show exactly what it is you're trying to do. For example, what exactly are the differences between the images? Are they different angles of the same character? Different expressions of the character? Both? The more images and variables you have to morph between, the more complicated this gets.

I'm not convinced that's a practical way to use Moho. Moho favors taking the opposite approach: using mesh warps to get more angles and expressions out of a single image or very few images.

To do this well involves deforming the pixels while also blending the colors/patterns intelligently and precisely, which suggests using optical flow tech that Moho doesn't have. Or maybe that's more sophisticated than what you're asking for? Using meshes or splines to morph between two or more different images can be done more realistically in a compositing programs like Fusion and After Effects, but even there it involves a lot of work and it's not an automated process.

Will give this some thought but it will help to seen an example of what you're trying to do.

[Greenlaw]

BTW, I'll still create an improved version of the rig and post it. I wasn't able to do that last night but maybe this evening.

Also...getting more intrigued about trying a multi-image setup...darn you!

[madrobot]

Also...getting more intrigued about trying a multi-image setup...darn you!

LOL! Sorry mate!
Welcome to a brand new rabbit hole you didn't even ask for!

[synthsin75]

Stoopid, quick example.


https://www.mediafire.com/file/u93ifoyk ... c.zip/file

[madrobot]

*applause*
Wes, everybody. Wonderful. Handsome. Excellent.
Missed you mate, great to be back in the soup. That is a fantastic test result and is very promising.
Thankyou very much for your time and effort.

[Greenlaw]

Ha, ha...stoopid and awesome!

Nice example, Wes! And thanks for sharing the file...looking forward to picking it apart tomorrow.

[chucky]

WOw, what a great idea, using one image and moving it...
Very clever Wes.

[Greenlaw]

Finally got to look at the project file. Yes, that's a very clever trick!

[Maestral]

Why indeed! Excellent one, Wes!