3D and 2D integration

General Moho topics.

Moderators: Víctor Paredes, Belgarath, slowtiger

Post Reply
Animator101
Posts: 66
Joined: Sun Jul 16, 2017 7:18 pm

3D and 2D integration

Post by Animator101 »

Is there a way to integrate 3D onjects or characters from 3D softwares and rig them in Moho? I tried to import .obj file from Poser to Moho but it didnt go well. I didnt even see it in my canvas. . Do you guys use and 3D obnects in Moho?
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: 3D and 2D integration

Post by Greenlaw »

I do 3D and 2D Moho integration, but not normally in Moho. At least not directly.

Typically, there are three ways I do this for both my own projects and the stuff I do at work.

1. The 3D CGI animation is rendered out first and then image sequences brought into Moho. You can then position your characters where they need to be. You might output the 3D FG and BG passes separately, and import them as separate Image Sequence layers in Moho. This is probably best when the camera is relatively static, otherwise, you'll need to match move in Moho as well.

2. Moho characters are rendered out and then mapped to cards in the 3D program. Normally, you'll animate the characters in place and then animate the card positions in the 3D program. This is best if the camera is moving a lot through the environment. Set your materials to full luminance and no diffuse, unless you really want the characters to look like 'cards' in the environment.You'll also want to set the cards to face camera on Y. You can also use X but it might look weird if the camera flies up and looks down.

My first 'Moho' short film Scareplane did some 2D integration within LightWave, a 3D animation program I use. In the second scene, I rendered Sergeant from Moho and mapped him to a card that was inserted in CGI cockpit. This is a very basic use of the technique. A more complex example: a while back, I worked on a scene with a 3D neighborhood and a camera flying 'drone-like' through the street; in this case, we animated all the 2D characters walking/performing in place and animated their card positions in the 3D program. This made the characters look fully integrated even with the complex camera moves.

3. Do it in comp. I normally render only the Moho elements from Moho and then integrate it with the 3D in compositing so I can apply special edge treatments and add shadow and lighting effects more easily, which can make the 2D/3D integration look more polished and credible. Some compositing programs like Fusion and After Effects have 3D environments, so you can do 3D setups there too. You can see examples of this in two shorts on my website: Scareplane and HLF. Scareplane was composited in Fusion and HLF was composited in AE. Both shorts leveraged the 3D environments of each compositing program for many scenes. The shorts also used Moho for some 3D camera and multi-plane effects, but 3D objects from outside sources were never actually imported to Moho.

Another reason to do it in comp: Many popular compositing programs these days have excellent 2D and 3D tracking tools, which make it easier to place the 2D animations inside 3D renders.

Tip: With method #2, unless you're going to composite in Moho, I suggest importing lower quality JPEGs for the Image Sequence layers. This will stream faster when scrubbing and creating animation previews, and not be used for final rendering. When you render, set up a Layer Comp for just the character animation and composite that with the 3D animation in your compositing program. You don't have to do that of course but you'll find better options for making the characters look integrated inside a compositing program than in Moho.
Animator101
Posts: 66
Joined: Sun Jul 16, 2017 7:18 pm

Re: 3D and 2D integration

Post by Animator101 »

greenlaw....I have to say that I am really impressed by thoroughness and comprehensiveness of your reply. I learned a lot from it even though I am still trying to understand your approach. Obviously first thing that I WON'T DO anymore is to import effects into Moho and try to have 3 explosions and smoke effects plus full blown animation run at the same time. Even my new 2017 MacBook Pro slows up during those tasks :)
That's what After Effects is for or Fusion like you said.

Second thing..I imported .OBJ from ZBrush 3D sculpting program and it worked fine.However I felt like Moho is having a hard time navigating in real time when I leave my 3D Option-Shading at SMOOTH. Is that too CPU demanding ? Should I use just TOON till I rig and animate my charater and than switch to SMOOTH before rendering or no ? IN case of course I want to end up with rigged and animated 3D mesh imported from 3D program and expecting it to kind of look like when in 3D program. Also what are "FG and BG passes" ?

Third thing...I would really like to know what did you mean by "Moho characters are rendered out and then mapped to cards in the 3D program".
How do you "map to cards". I decided to use After Effects for compositing for now. I got a trial so I guess I will be doing most of this in AE instead Moho like I tried so far.

I really appreciate you taking time to explain all of this.
User avatar
david_macrae
Posts: 9
Joined: Fri Jan 30, 2015 8:44 am
Contact:

Re: 3D and 2D integration

Post by david_macrae »

@Greenlaw I also always comp my elements usually in Apple motion and FCP X but recently I got Fusion and am slowly learning it/ One thing I am interested in is what you actually do for your your edge treatments. I think this is one of the most crucial steps to get a good natural blend between 2d and 3d.
Image
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: 3D and 2D integration

Post by Greenlaw »

Sure, there's a lot of things you can do in Fusion for edge treatment. The techniques may seem like overkill to some but if you're working in HD or higher, it does make a difference.

Here's a fairly easy trick for softening edges:

1. Add a Load node and import a Moho character with alpha.

2. Add a Filter and set it to Sobel. This makes an outline for the edges, and if you look at the alpha, you'll see that it has created an outline mask for you.

3. Feed this into a Bitmap Mask*.

4. Blur the mask slightly. You can do this either in the Mask node with Soft Edge (use a very tiny value) or add a Blur node. This will give you a slight fall-off.

5. Now, add a Blur node after the original Loader and feed the Bitmap Mask (or the Mask's Blur node if youre using one,) into the Blur node. Now you can soften the edge of the image to better integrate it with any background image.

*(While it's possible to feed the image directly into another node as a mask, it's better to do this through a Bitmap Mask node. This is especially important when working in higher bit-depths. That's not currently an issue with Moho renders because Moho is limited to 24-bit, but I'm hopeful that Moho will support higher bit-depths in the future. Until then, may as well get into the habit of doing it right.) :)

If that's too many steps (it really isn't once you get the hang of it,) you can turn this into a macro. (I'm not sure macros are supported in the free vesion though.)

Anyway, the above can serve as a foundation for many more advanced tricks. For example, you can create light wraps by off-setting another copy of the bitmap masks, use that to gain one side of the render, and then feed in a blurred copy of the background to natually tint the lighting effect. Of course, this can be combined with the edge softening described above.

This is the kind of thing I've been doing for CGI and live action integration for many years and it works well for Moho too. IMO, Moho's antialiasing is okay most of the time but sometimes I need to apply tricks like this to improve the edge quality. For lighting effects, you can using compositing to enhance what Moho offers natively, or you can use masking tricks as described to create more realistic effects. If the lighting needs to be interactive, I find that's a whole lot easier to do in comp than directly in Moho. (I may use either method. It depends on the bg environment and the kind of lighting that's called for in the scene.)
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: 3D and 2D integration

Post by Greenlaw »

Animator101 wrote:...Second thing..I imported .OBJ from ZBrush 3D sculpting program and it worked fine.However I felt like Moho is having a hard time navigating in real time when I leave my 3D Option-Shading at SMOOTH. Is that too CPU demanding ? Should I use just TOON till I rig and animate my charater and than switch to SMOOTH before rendering or no ? IN case of course I want to end up with rigged and animated 3D mesh imported from 3D program and expecting it to kind of look like when in 3D program. Also what are "FG and BG passes" ?

Third thing...I would really like to know what did you mean by "Moho characters are rendered out and then mapped to cards in the 3D program".
How do you "map to cards". I decided to use After Effects for compositing for now. I got a trial so I guess I will be doing most of this in AE instead Moho like I tried so far.
As mentioned earlier, I don't bother with bringing 3D into Moho. It's a hassle and there are too many limitations (like no 3D lighting tools.) IMO, it's better to cel-shade in your 3D animation program and render the CGI separately for compositing.

FG/BG is Foreground and Background. In this context, 'passes' essentially means 'layers'. You can use Moho's Layer Comps panel to break out your passes. For example, you might have a layer comp for Background, Midground, Characters, Props, and Foreground. You can even combine layers in Layer Comps to use as special control masks for your compositing program.

With Moho, I typically have 2 or 4 Layer Comps but I've had situations where I had to output dozens of Layer Comps for compositing.

Re: Mapping to cards, this just means applying your Moho character renders as a texture map to a flat, rectangular mesh in your 3D animation program. Once you have your 'card', you just place it in the 3D environment facing the camera. Most 3D animation programs allow you to constrain the rotation of the card to always face camera so you only need to animate the position, not the rotation.

From there, you can render it all in one pass from your 3D program, but I prefer to render the 3D 'cards' animation as a seaprate pass for compositing. There much you can do to improve a render in compositing and it's much easier and faster to address changes/fixes in post.
User avatar
david_macrae
Posts: 9
Joined: Fri Jan 30, 2015 8:44 am
Contact:

Re: 3D and 2D integration

Post by david_macrae »

@Greenlaw Thanks for that in information. I look forward to incorporating it into my workflow. I think you do need the pro version to write macros. I just went to their website and see that they recently released a new version 9 with a lot of great tools. Also the pro version was lowered to $299 so that is a great deal I will hopefully be getting soon.
Image
Animator101
Posts: 66
Joined: Sun Jul 16, 2017 7:18 pm

Re: 3D and 2D integration

Post by Animator101 »

A lot of useful info here. I will try to follow even though I am far from that level. I have another question...if you want to import a character made in 3D application into Moho and rig it, how do you make Moho render textures and paint applied previously in the 3D app ? Is that supposed to be a part of .OBJ file or it has to be entirely different export ?
Thank you
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: 3D and 2D integration

Post by Greenlaw »

Oh, yeah, I forgot about that. I agree, at the new $299 price, Fusion is an absolute steal.

When I worked for Rhythm and Hues, The Box (the team I was with) used Fusion in every commercial and video game cinematics job that came our way for over 12 years. More recently, I used Nuke on a job and, even though the two programs share similarities, to me, Nuke seems surprisingly unpolished compared to Fusion. And now, Fusion has become a lot cheaper than Nuke. Earlier this year, I upgraded my old Eyeon Fusion 6.x to BMD's Fusion 8 Studio for about $900, and even then I thought that was a good deal. (I guess it's all relative: originally paid around $2000 for Fusion 4 way back when Fusion still retailed for about $5000.) Well, okay, I admit I was taken aback to hear the price had dropped so low so soon after I made my purchase last spring, but then I learned my Fusion 9 Studio upgrade would be free so I didn't feel so bad about it. :)

$299 for a full non-expiring license with unlimited render nodes, optical flow tools, a 3D camera tracker, Primatte, scripting, plugins support, etc., yes, that's a bargain.
Last edited by Greenlaw on Sun Aug 27, 2017 7:03 am, edited 2 times in total.
User avatar
Greenlaw
Posts: 9192
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: 3D and 2D integration

Post by Greenlaw »

Animator101 wrote:...if you want to import a character made in 3D application into Moho and rig it, how do you make Moho render textures and paint applied previously in the 3D app ? Is that supposed to be a part of .OBJ file or it has to be entirely different export ?
If the .obj has uv maps, and the texture maps were painted for it, the textrues should show up in Moho. But that's pretty much it. Moho doesn't support any other material settings, the 3D is limited to basic transforms (no deformations,) and, AFAIK, 3D lighting tools do not exist in Moho.

I would suggest just trying it out for yourself. Depending on your expectations and project needs, none of the above may matter. But if you already use a 3D animation program, you may be disappointed.

That said, Moho apparently has some kind of link with Smith Micro's Poser. I've never used these two programs together but Poser does have everything I mentioned above, so maybe this link overcomes some of the limitations? I don't know...just putting it out there. Other users here will know a lot more about this than I do, so hopefully they'll jump in with better info.
chucky
Posts: 4650
Joined: Sun Jan 28, 2007 4:24 am

Re: 3D and 2D integration

Post by chucky »

I have tried it but it is very limited to basic prop stuff or basic internal extrusions. Don't bother with anything other than a toon render.
I have imported things like city scape , but there is a problem with lighting being locked to camera so even basic shading is pointless.
Successful integration is only possible for other than the most simple prop items , built and animated externally and either externally comped or imported renders.. see this example of 3d integration, exteral render motion tracked and combined with 2d elements inside Moho.

Post Reply