Ways to animate a mouth from a photo?

Wondering how to accomplish a certain animation task? Ask here.

Moderators: Víctor Paredes, Belgarath, slowtiger

Post Reply
Welsh Jester
Posts: 44
Joined: Mon Jun 30, 2014 1:41 am

Ways to animate a mouth from a photo?

Post by Welsh Jester »

I was wondering if anyone knows of any ways to best animate a mouth from a photo? I know there's Smart Warp, but what i want to do is cut out heads from photos and animate the mouth more easily. I know it can be done Monty Python style, but i am more after a different style, i guess kind of like in that cartoon Angela Anaconda, if anyone is familiar with it. I'm not aware of any other cartoons that use that particular style of mouth animation.

The problem is that to get it to look decent i would need actual photos of different mouth shapes, and it seems that is how they did it for that cartoon. Are there any other ways of being able to do this in an easy/lazy way? If i use a cartoon style mouth then it obviously isn't going to look right placed on top of a real persons mouth.

This is just something i want to mess around with for fun, but i would appreciate any ideas on this.

Thanks in advance.
User avatar
slowtiger
Posts: 6067
Joined: Thu Feb 16, 2006 6:53 pm
Location: Berlin, Germany
Contact:

Re: Ways to animate a mouth from a photo?

Post by slowtiger »

If you have a camera, no matter whether still or video, just act out the mouths yourself or ask a friend.

Some years ago I did a quick test for a music video, used some mouths found in the net:
AS 9.5 MacPro Quadcore 3GHz 16GB OS 10.6.8 Quicktime 7.6.6
AS 11 MacPro 12core 3GHz 32GB OS 10.11 Quicktime 10.7.3
Moho 13.5 iMac Quadcore 2,9GHz 16GB OS 10.15

Moho 14.1 Mac Mini Plus OS 13.5
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: Ways to animate a mouth from a photo?

Post by Greenlaw »

Years ago, my daughter and I were messing around with CrazyTalk. Here's one of our tests:



It's creepy as heck but, more importantly, it was created from a still image and the animation was fully automated. I think that's about as lazy as you can get for this sort of thing. :D

If you don't mind a little more work, it would be fairly easy to do this with SmartWarp in Moho. Just save the mesh key poses for as Morph Actions, mark your timeline and click away.

You could also add the poses to a single SBD, which would be easier to animate but you'll lose the interpolation because you need to animate in Step mode. That might not be important though--interpolated keys for lip sync tend to look kinda mushy anyway.

Another method would be to copy the region around the mouth in Photoshop and use the Animation Timeline to paint/edit the different mouth poses as a sequence. When you're done, render out the sequence and load it into Moho as an Image Sequence layer. Finally, convert the Image Sequence layer into a Switch Layer, and set up an SBD for it.

Another lazy method: shoot a video of somebody mouthing all the phonemes, and save out key frames for a Switch Layer.

If you're super lazy, you can always just Space Angel or Clutch Cargo it. :D
Welsh Jester
Posts: 44
Joined: Mon Jun 30, 2014 1:41 am

Re: Ways to animate a mouth from a photo?

Post by Welsh Jester »

Thanks for the examples and advice guys.

Would Smart Warp be the best solution for images taken of random people off the net, even if slightly side facing? The only thing is that an image really needs preparing first in an image editor, like if the person has his/her mouth closed, then it's better to make make a small gap in the mouth before importing to moho to give it a smooth look. Simply setting up a mesh around an already closed mouth and making a gap in the lips in moho gives a very rough look.

I have noticed that switching GPU acceleration off has an effect on the position of images that have a smart warp layer set as well. For example if a mesh is set up and a layer is set to that mesh, if you turn off GPU acceleration and switch from frame 0 to 1 then the image will jump position a little, so it screws up the mesh positioning. I'm assuming that this isn't just me and the same for everyone else right?

Guess maybe actions is the best, I'll be taking heads and shoving them onto vector bodies, it still seems like it would be quite a bit of work just to set up all the mouths though. And for other stuff like if i wanted some teeth sticking out from the mouth, then i would have to use masking right? Hmm..
Welsh Jester
Posts: 44
Joined: Mon Jun 30, 2014 1:41 am

Re: Ways to animate a mouth from a photo?

Post by Welsh Jester »

After some messing around i did manage to get a result, it's a little comical looking but it kind of works. Link here https://ufile.io/90b4z

Though it's not really all that friendly to work with since there's no keyframes on the group layer. The tongue seemed to want to appear only after i clicked on the layer first too. So yeah it has limitations i guess but i don't know of any other ways of doing it, i guess it's just because of the way the smart warp works. With the head being all 1 layer and the mouth being part of that layer it's kind of awkward to get some mouth poses right.. like f, since the teeth can't go over the lips.
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: Ways to animate a mouth from a photo?

Post by Greenlaw »

Sorry, I haven't looked at the link yet...still at work....but will take a look tonight.

Just a thought: if you setup your mesh properly, you can sort the stacking order and animate it. In other words, it is possible to have the teeth move in front of or behind the lips as needed.

To see how this works, check out Victor's stone dragon example that comes with Moho 12. The 'rolling' wings are a good example of how polygon sorting works.

There are other ways to handle this too, like using separate mouth groups within the face group and switching between them. You can tie this to an SBD and control the lipsync from the same layer as the other bones. (If you go for this approach, remember to enable Use Previous Key and Step Mode.)
User avatar
hayasidist
Posts: 3493
Joined: Wed Feb 16, 2011 8:12 pm
Location: Kent, England

Re: Ways to animate a mouth from a photo?

Post by hayasidist »

a smart mesh controlling an image also crops the image, so it's possible to work on some segmentation of the image -- e.g. one crop for mouth and one for the whole head -- so that you can maybe swap layer order for mouth / teeth to get teeth in front of lips (and there are other approaches to get bits to be sometimes in front and sometimes behind)
Welsh Jester
Posts: 44
Joined: Mon Jun 30, 2014 1:41 am

Re: Ways to animate a mouth from a photo?

Post by Welsh Jester »

Greenlaw, i checked the stone dragon and all i can see is that the wings are all on separate layers. I'm talking just 1 image (head) layer, and sorting another layer (teeth) to be behind the lips and then sometimes in front of just the bottom/top lip. It doesn't look like it's possible like that to me.

hayasidist, I'm not completely sure what you mean at the minute but I'll try and give it a look at later.

Thanks for the input.
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: Ways to animate a mouth from a photo?

Post by Greenlaw »

Yes, the feather images are mostly separate layers but you need to look more closely at what's happening there. Observe the actual Mesh layer for the nearest folding wing. This is a single layer and it appears to roll over itself like it's made of cloth. This possible because a specific stacking order has been assigned to the polygons by the artist to keep the nearest part of the wings is in front and the farthest stays behind.

If you have trouble seeing it, try this:

1. Open Wing 1 and Alt-Click on Objecto 4 to hide all other layers.
3. Click on the Mesh Wing 1 layer in the same group.
2. Click Play or scrub.

(Sorry, I meant to include a screen cap but TinyPic appears to be down at the moment.)


You can apply this same technique to the stacking order of the lips, teeth, and tongue, all within the same layer of polygons or mesh. You can even change/animate the sorting order of the polygons by enabling Animated Shape Order, and keyframing this in an Action.

For example, moving the lower lip under the upper teeth to make the 'F' sound, and then pulling the lower lip over the upper lip to make a pout expression. If the lower lip image is sectioned in they're own polgyons, you can move the lip shapes up or down the stacking order relative to other filled shapes in that mesh, even when they're all part of the same single image.

Note that I'm describing a fairly advanced technique. Since we started off discussing 'easy' methods, maybe we don't want to go down this path. :)

(Personally, I would just make a switch group containing multiple mouth groups, and use normal masking. You wind up with a lot more layers and groups but, IMO, it's a lot more flexible to rig and animation changes this way.)

Hope this helps.
Last edited by Greenlaw on Thu Aug 10, 2017 4:26 pm, edited 1 time in total.
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: Ways to animate a mouth from a photo?

Post by Greenlaw »

I should have noted that this can be done only in Pro. You can't create a custom mesh in Debut.
Welsh Jester
Posts: 44
Joined: Mon Jun 30, 2014 1:41 am

Re: Ways to animate a mouth from a photo?

Post by Welsh Jester »

Greenlaw, forgive me if this sounds stupid but I'm still not really getting it :?

The polygon sorting i know how to do with the mesh & the imported image, but when i make another vector (say teeth) to another layer, i can't see a way to partially hide that in with the image layer. The polygon sorting is only working for the imported image. To me, the vector needs to also be on the same layer as the image :?: and that doesn't seem possible.

For the other way that you mentioned using a switch group, i see that there's more than 1 way of doing that too right? First is using 2 switch groups, one with multiple image layers and the other with multiple meshes. Second involves cuts/crops or whatever, and just the 1 switch group?

I can't even believe I'm having so much trouble with this when it's probably really simple, i usually find my own way around problems using software..

Maybe my brain has stopped working :lol:
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: Ways to animate a mouth from a photo?

Post by Greenlaw »

Can you post some screen caps or examples of exactly what you're trying to accomplish? Some of the descriptions so far seem conflicting and I'm not sure I follow you.

Just for clarity:

You mentioned Angela Anaconda at the head. I finally got around to looking at some the old videos on YouTube. Oh, yeah, I remember that! :)

If you step through the footage, you'll see that it's all done with still photo elements, no 'warping' involved. You can replicate that using basic switch layers and a series of photos. You mentioned wanting a 'lazy' way to do this, and the 'layziest' is probably to shoot all the faces and mouth shapes, chop them up in Photoshop, and then place the elements in separate switch layers. At minimum, you'll want a single Face switch with all the possibilities, but what would look better is an Eyes switch and a Mouth switches over separate static 'head' layer would look better, and it would be exactly like Angela.

If you want to add some nice subtlties, use a mesh and link all the elements to the single mesh, and use the Magnet tool or pin bones to warp it. But that's already going way beyond what you see in the Angela footage.

If your photo source is limited, you can paint variations of the head features in Photoshop. Liquify in Photoshop is awesome for this, especially now that they've added tools specifically for altering the human face. I've done this sort of thing with painted human faces at work, and it saved me from painting too many variations from scratch. Again, it's a little more work, but if you're not going to shoot enough actual photos in advance, it's a good alternative approach.

If you want to do it 'right', you'll want to break the face down into more elements: separate switch groups for each eye and the mouth, and and breakout the eyebrows with their own bones or mesh. You might even make separate the nose for 'wrinkling', etc. Of course, now were getting a lot less 'lazy' about the possibilities...but it's still not terribly complicated stuff.

And there are other cheap tricks you can use to make it look much smoother than Angela. For example, if you break out the mouths to their own Switch Layer, you can put a lot of life into the lipsync by squashing and stretching the group layer. Just set the origin to the upper lips, and then you can credibly transform the group as a whole. I recommend doing the S&S as a second pass, after you've completed the lips sync pass--it will get too complicated and messy to do both at the same time. Oh, and don't forget to add a pin bone to the chin so you can drop the jaw appropriately. Both, the Lipsync switch and the jaw-mover can be linked to Smart Bone dials to make it easier to animate, and if you set it up right, you'll still have flexibility to add your own manual tweaks on top of it. But, yeah, we've gotten pretty far away from 'lazy' at this stage. :P

Just wondering, which version of Moho are you using? Pro or Debut? Only Pro allows you to create and edit a custom mesh. A single mesh layer can be shared by any number of vector or image layers, so that shouldn't be a limitation. However, where you place the mesh layer may prevent some layers from accessing it, so make sure you're using the mesh correctly.

Much of what I think you're describing can be accomplished a number of ways: Using Switch Layers with static images, using animated meshes and/or bones to deform static images, animated layer order, animated shape order (texture filled shapes all in the same layer,) masking tricks, or a combination of any of these techniques. All the tools are there, it's a matter of knowing what you want to do with them.

Earlier, I mentioned Liquify in Photoshop. If you have After Effects, Liquify also exists there and it's fully keyframeable. I sometimes use this to add subtle (and sometimes not so subtle) deformations to my Moho renders. It's very useful for getting a little more expressiveness out of limited image resources.

Because there is a wide range of tools and possibilites, I suggest designing a character with specific techniques in mind, and then creating a simplified version of the concepts you wish to test. Once you've proven your method, apply what you've discovered or learned to your finished character. This is pretty much my workflow for every new character and design, whether it's for my own stuff or a production I'm assigned to work.
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: Ways to animate a mouth from a photo?

Post by Greenlaw »

In writing the above, it occurred to me that, for shape order changes to affect the art, you may need to use Image Texture fills sort it all in a single layer. This way the art and the Mesh exist in the same layer. It's a good technique but it may limit how you can use it with switches.

Well, unless you want to make many copies of the mesh, each with a different Image Texture fill. Just a thought...I don't necessarily recommend that if you intend to use a lot of mouth/face variations. But then again, with the flexibility you gain with Smart Meshes, you may not need too many variations to cover a broad range of expressions.

Again, not a 'lazy' way to do this but with a little planning, it wouldn't be that difficult to set up either.

I think that's the key word here: planning. Nothing I've described so far is really that difficult to do if you have a clear idea of what you want to see in your animation and plan the steps to accomplish it. I would work out the design and concepts on paper first before spending too much time in Moho. With out a good plan, you may wind up just floundeing from one technique to another and not really getting anywhere.

Hope this helps.
User avatar
hayasidist
Posts: 3493
Joined: Wed Feb 16, 2011 8:12 pm
Location: Kent, England

Re: Ways to animate a mouth from a photo?

Post by hayasidist »

Welsh Jester wrote:
hayasidist wrote:a smart mesh controlling an image also crops the image
hayasidist, I'm not completely sure what you mean at the minute but I'll try and give it a look at later.

Thanks for the input.
if you have an image and you create a smart mesh that covers (e.g.) only the eyes then the rest of the image won't be rendered - you can use the mesh just to control the eyes. make a copy of the image and put it under the mesh-controlled segment to get the cropped bits. like this crude example https://www.mediafire.com/file/ylz4x7pt ... xample.zip
Post Reply