Stylit convert 3D object into a hand drawn style drawings

A place to discuss non-Moho software for use in animation. Video editors, audio editors, 3D modelers, etc.

Moderators: Fahim, Distinct Sun, Víctor Paredes, erey, Belgarath, slowtiger

Post Reply
ruscular3d
Posts: 378
Joined: Sun Dec 26, 2010 12:18 am

Stylit convert 3D object into a hand drawn style drawings

Post by ruscular3d » Tue Sep 18, 2018 5:11 am

https://www.youtube.com/watch?v=F6ex4pogilA

This could be great for making new background setting for the MOHO animation.
User avatar
Maestral
Posts: 327
Joined: Tue Jan 27, 2009 5:44 pm
Location: Belgrade, Serbia

Re: Stylit convert 3D object into a hand drawn style drawing

Post by Maestral » Tue Sep 18, 2018 11:22 am

This one (since there is a lot more these guys have done) is very interesting but I have not figured out how to actually apply it!? Have you?

p.s.
Does anybody know how to make aquarelle convincingly animated, preferably in 2D? Mesh would, more or less, solve the mapping/texturing but StyLit offers these fine changes of light and shade through the movement and I`m looking for something similar.
ruscular3d
Posts: 378
Joined: Sun Dec 26, 2010 12:18 am

Re: Stylit convert 3D object into a hand drawn style drawing

Post by ruscular3d » Tue Sep 18, 2018 5:11 pm

I am not seeing anywhere that being applicable to us now, seems to be in the demo stage for now. I dont even have a price info either.
User avatar
Maestral
Posts: 327
Joined: Tue Jan 27, 2009 5:44 pm
Location: Belgrade, Serbia

Re: Stylit convert 3D object into a hand drawn style drawing

Post by Maestral » Tue Sep 18, 2018 10:21 pm

I`ve seen a 3D still life scene (fruits, baskets, cloths...) on which some style from StyLit was applied. Looks great but than again, no details on "How it`s made" ,)
dkwroot
Posts: 659
Joined: Thu May 02, 2013 6:56 am
Location: USA
Contact:

Re: Stylit convert 3D object into a hand drawn style drawing

Post by dkwroot » Wed Sep 19, 2018 2:29 pm

I'm guessing that it's using a constitutional neural network that's been trained for style transfer. The program renders the 3d scene like normal, then samples the style created by the artist, then applies it to the render using the neural network.

Style Transfer Info: http://genekogan.com/works/style-transfer/

More info: https://medium.com/tensorflow/neural-st ... 541ac31398
User avatar
Maestral
Posts: 327
Joined: Tue Jan 27, 2009 5:44 pm
Location: Belgrade, Serbia

Re: Stylit convert 3D object into a hand drawn style drawing

Post by Maestral » Wed Sep 19, 2018 3:52 pm

Your comment made me look through their full (patented) paper.

"The deep neural network approach of Gatys et al. [2015] strongly depends on visual patterns used during the training phase. In our scenario this approach completely fails (Fig. 10h); without proper guidance it is impossible to recover a meaningful assignment between the source and target features."

Later on, they are mostly relying on LPE channels and even gave the recommended PC`s specs along with rendering time estimation for a 1 megapixel image (3-15mins, respectively to HW configurations). That made me think some standalone app could be found. Alas, the ending of the paper brought a hint - "Sponsored by Adobe" ,)

StyLit org provides Windows (only) demo where one may play with a printed template and painting their own style which would be then applied onto existing models/examples, within the demo. I think I saw a few dependencies along the way but can`t be sure will those be really necessary.

Although this sounds like 3D oriented sw, actually these guys already made it available for stylizing animation/video with a peculiar attention on minimizing frame by frame differences and preservation of character/BG integrity through the movement - while still having the desired style applied properly.
Post Reply