Antialiasing/pixelation problem in brushes

[Daxel]

I haven't paid much attention to brushes until now (Moho14) but I think this happens in previous versions aswell. The new line boil feature just made brushes far more interesting for me as I think it's a very convincing and smart way to fake a FBF feeling, specially being able to set it to only boil when the drawing is in movement.

In this file I hand draw a Luffy and then tried many different brushes with different thickness to understand what the problem was and if I could change something to make brushes quality acceptable for production, but I failed. The problem, as the title says, is pixelation, poor antialiasing, that is far more present on thin lines but as you can see there are pixelated parts even on the thicker lines, just in less parts. It is common to see pixelation only in one side of the stroke but not on the other side of it, which is strange and may give us some information about the problem.

You can see the first column (normal vector line) has good antialiasing for comparison with the rest.

Rendered image (1080p). Try to see it at real 1080p size so there is no scaling that can affect the end result:
https://1drv.ms/i/s!AiyKCw6Xey5Tgpt8n2p ... g?e=XBha3h

Moho file:
https://1drv.ms/u/s!AiyKCw6Xey5Tgpt6ZJY ... A?e=2Oh1uB

Edit: changed both links to a new version that has a white background instead of a transparent one.

[Daxel]

Changed both links to a new version that has a white background instead of a transparent one so it's easier to visualize.

[hayasidist]

Some observations. (And a long way from anything that might "resolve" pixilation etc issues)

Brush "stamps" are bitmaps and usually square (e.g. 512 px * 512 px) but they're not forced to be. I've experimented with varying sizes from 8px * 8px up to 1k*1k, with a few "oddball" sizes (e.g. 256 *10).

First observation (using "align with curve") is that Moho tries to align the stamp with the direction of the curve, but if the curve changes direction within the bounds of the stamp, the bitmap (unsurprisingly?!) does not "bend". This gives some of the edge artefacts seen around curve bends / corners especially from overlapping stamps

Second observation is that Moho tries to fit "the right number" of stamps along a segment (and re-starts the "fitting" process at the start of each segment). This can lead to gaps / overlaps near mesh points.

Third observation is that brush spacing has (sensibly and intentionally) a significant effect on how the rendered stroke appears. Just fiddling with this number in Daxels' example file changes the image. (e.g. change the spacing in 8px MC Chunky to 0% and the image looks very similar to other 0% spacing brushes)


The following are in the realm of "I think this is what's happening", but I'm not totally sure.

a) If I have a brush that is not square (e.g. my 256*10) then Moho "squares" it by replicating the short dimension to fill the missing space. I can kinda see a rationale for this in situations where there's NOT a requirement to "align with curve".

b) If I have a brush that uses only (say) the middle third of a square brush (e.g. a 1k*1k bitmap that is transparent in the top and bottom thirds, with strokes restricted to the middle third) in the "align with curve" situation this "works", but the Brush Spacing needs to be set very low to get a continuous stroke [and then brush behaves as in the above observations re bends and segments].


===

I'm not sure where to go with this. Moho's approach to deploying brush stamps along a path doesn't seem to be out of step with the few other bitmap-based programs I've tried (and, arguably, is more sophisticated as it does include all the goodies we now have in 14). A part of me imagines a "mini curver" over each stamp that actually distorts the bitmap to follow the bend in the vector line, but I can't see that being a sensible approach for all manner of reasons including performance ....

anyone have any good thoughts?????

[Daxel]

Very interesting thoughts.

I tried to make a custom brush with a very small size to see if maybe it was the texture scaling that failed, and it was one of the brushes with best behaviour in thin lines but still had antialiasing problems.

I agree that brushes in Moho are actually very cool and rich in features, I feel it would be a waste not being able to use them for this strange render issue. Well to be fair some of them are fine in pretty thick sizes but yeah most productions don't use lines that thick.

[Greenlaw]

About your antialiasing /pixalated issue... I don't think I see the same problem. Do your have an example file - that others can check?

For me, there used to be a problem with brush AA, but that was way back in ASP 11, and I think the issue was fixed by the time Moho 12.5 was released.

In ASP 11, there were maybe two brushes that had minimal noise issues so I tended to stick with those, and the main solution for me was to use the Extra Smooth images render option. This option renders the output at double size and then scales it down to the target size to improve the overall AA quality. Technically, this option increases the render time by 4x, but it's totally worth it for me. (Moho always renders pretty quickly anyway...well, certainly compared to the 3D animations and VFX I'm used to rendering.)

[Daxel]

For me, there used to be a problem with brush AA, but that was way back in ASP 11, and I think the issue was fixed by the time Moho 12.5 was released.

I don't know but I see very clear pixelation in the examples I linked. You don't?

In ASP 11, there were maybe two brushes that had minimal noise issues so I tended to stick with those, and the main solution for me was to use the Extra Smooth images render option. This option renders the output at double size and then scales it down to the target size to improve the overall AA quality. Technically, this option increases the render time by 4x, but it's totally worth it for me. (Moho always renders pretty quickly anyway...well, certainly compared to the 3D animations and VFX I'm used to rendering.)

The png I linked was rendered with Extra Smooth Images activated in the export options, but it doesn't solve the problem. There is a Higher Render Quality option inside layer settings/image but only for image layers.

[teienkawi]

i just experimented trying to use a svg as a brush its pixalated like it was made in paint but it doesnt have the kinda grey blur/fade that even using none for a brush has. i dont know how to upload images here to show the example


i used inkscape to make the svg

[hayasidist]

Not sure moho can handle a .svg as a brush (just tried and it just gave me a black square rather than the blob I wanted).

I'd been using PS to make brushes, then it occurred to me that anything that makes .pngs would work - so I used moho to make a multi-brush -- project dimensions 512*512; 25 frames; a base stroke plus particles to give "judder". Render no AA and drop the resulting folder with its .pngs into custom brushes. Worked as expected (along with all the artefacts etc as in my earlier post).

[Rai López]

I agree the way brushes render, even though they do it much better now than it used to do time ago, has always been a little harsh compared to vectors in terms of antialiasing, over all under certain situations (thin lines, i.e.), and I've always thought there should be some room for improvements there. Said that, and talking about the Sketchy effect, I've used it at times to somehow counteract this problem. Again, to proof a point:



Make sure you can maximize the window enough to can really see the image in real size, or just download, and pay special attention to thinner lines. The reason behind not applying it to the 8-px line heads, is in such case it start not being necessary IMO, or IOW, from certain line thickness on you may start having more to lose (overall regarding brush texture) than to really gain, since aliasing artifacts start becoming much less evident.

Here it is the uncropped image for allowing direct comparison with Daxel's: pixelated brushes tests luffy with background_00001_sketchy-ed_1px_rl

And, just in case it help, the modified moho file: pixelated brushes tests luffy with background_rl.moho

For me, the key always has been try to use as less amount of sketchiness as possible (accordingly to line width), in order to avoid/minimize little details and brush texture losing. Of course, the best outcome would be that little-sized brushes AA were improved at some point in the future, but there it is for what it could worth... or if it's only to somehow illustrate that if a Shape Effect can add some kind of extra-antialiasing, there may be already something under the current system that could make it better after all. Cheers!

[teienkawi]

i looked up how to post images but not sure if i got it right so at the bottom ill put the original links in case these 2 dont work


heres the example of my experiment earlier


https://www.dropbox.com/scl/fi/kuk1lzk3 ... 40os&raw=1



with this svg brush in the brush picking window and in the styles window for the selected brush its just white

but it still draws its really just a 512 x 512 black circle

https://www.dropbox.com/scl/fi/za4hhofr ... saad&raw=1

im keep experimenting with it cuz now its not working so good for actual use


(result of experiment)
image1 https://www.dropbox.com/scl/fi/kuk1lzk3 ... 740os&dl=0
(showing of the svg brush in the brush pickerwindow and style window)
image2 https://www.dropbox.com/scl/fi/za4hhofr ... esaad&dl=0

[hayasidist]

by teienkawi » Thu 28 Sep 23 0716
....this svg brush in the brush picking window...

It does look as though svg (a vector graphic standard) can't be used as a moho brush (which expects a bitmap)

[Daxel]

Said that, and talking about the Sketchy effect, I've used it at times to somehow counteract this problem.

That's very interesting! I'm playing with that now but it's hard to control in the real projects I'm testing it, mostly because they have different line thickness (line thickness tool) under the same line thickness (the one on the style) so with the same amount of sketchy effect some thin lines actually dissapear.

with this svg brush in the brush picking window and in the styles window for the selected brush its just white

but it still draws its really just a 512 x 512 black circle

Wow I didn't know that was possible. I wonder if it's actually taking the SVG shape or just falling back to some default thing.


Do you guys think there is a way to use the line boil on normal vector lines? I notice there is a line boil option in the project settings now, but the official manual doesn't explain what it does.

[Greenlaw]

i looked up how to post images but not sure if i got it right so at the bottom ill put the original links in case these 2 dont work...

Wrap the edited link with an img tag, like this...

Code: Select all

[img]https://www.dropbox.com/scl/fi/kuk1lzk31xseatiw74vou/svg-brush-example.png?rlkey=ekg1ay7jmkfeuk4ipssf740os&raw=1[/img]

The easy way it to select the link and click the image button.

The result in a posted message looks like this...



Hope this helps.

[hayasidist]

Wow I didn't know that was possible. I wonder if it's actually taking the SVG shape or just falling back to some default thing.
Do you guys think there is a way to use the line boil on normal vector lines? I notice there is a line boil option in the project settings now, but the official manual doesn't explain what it does.

I am of the opinion that svg will not work as a moho brush... viewtopic.php?p=213899#p213899
I'm guessing here - but it looks as though moho doesn't find anything it can use as a brush stamp (i.e. no bitmap) so just defaults to all bits -- hence the square

[teienkawi]

Green law:
thank you Green law
im new to posting on forum boards ive been reading them but this is pretty much the start



daxel:
"Do you guys think there is a way to use the line boil on normal vector lines? I notice there is a line boil option in the project settings now, but the official manual doesn't explain what it does."


i thought of a little work around for a sort of line boil for normal vector points it could be a bit of a hassle but the idea is to make a reference of the original and then put the layer setting noisy outlines animated on the reference layer but keep the original without the layer setting. then put those 2 layers in a switch layer so it can be switched between moving and not moving. this way isnt very good its been more of a experiment