Limited Codecs in Mac OS

General Moho topics.

Moderators: Víctor Paredes, Belgarath, slowtiger

Debo
Posts: 63
Joined: Tue Feb 26, 2008 11:26 pm

Limited Codecs in Mac OS

Post by Debo »

This is something that bugs me. I was an AS 5.6 user, but as that version doesn't work under Mac OS 10.8, I have to switch. So I'm trying out 9.1 now.

When I started using Anime Studio, I could choose between a multitude of codecs. In 9.1 however, there's only a few, Animation being the "best" one. Whereas I actually prefer to render my files uncompressed. If possible, without any codec.

The problem is that when I import QT files that have the Animation codec into Final Cut Express, the quality takes a hit. The H.264 codec is even worse. It's especially noticeable when a scene has gradients: they always show grainy banding. Uncompressed files don't have this problem.

Is the Animation codec really the best AS can do for Mac users?
User avatar
slowtiger
Posts: 6067
Joined: Thu Feb 16, 2006 6:53 pm
Location: Berlin, Germany
Contact:

Re: Limited Codecs in Mac OS

Post by slowtiger »

1. Codecs are always provided by the operating system, not by AS. AS only can use what's installed.

2. For file transfer between programs, like from AS to FinalCut, you always go uncompressed. I use PNG codec with millions+ colours because I need Alpha as well. Only finished films I render out of FinalCut in a lossy codec like h264 if I need small files for internet.
Debo
Posts: 63
Joined: Tue Feb 26, 2008 11:26 pm

Re: Limited Codecs in Mac OS

Post by Debo »

That's something I come across regularly at the forums but don't really get: how do you import .PNGs into FCE so that they form a sequence?
User avatar
slowtiger
Posts: 6067
Joined: Thu Feb 16, 2006 6:53 pm
Location: Berlin, Germany
Contact:

Re: Limited Codecs in Mac OS

Post by slowtiger »

No, not as image sequence - just export a Quicktime .mov with PNG codec.

Finalcut is quite clumsy with image sequences, but is capable of doing so. Have a look in the manual, it's all in there. Funnily enough in the manual they don't list PNG as a supported file format, yet I was able to import PNGs all the time.
Debo
Posts: 63
Joined: Tue Feb 26, 2008 11:26 pm

Re: Limited Codecs in Mac OS

Post by Debo »

I'll try that. You mean export with the .PNG codec from AS, right?

Still i don't understand how, in FCE, I can choose from several different codecs, among them Uncompressed 10-bit 4:2:2, but I can't in AS. If it's in FCE, it's installed, surely?
Debo
Posts: 63
Joined: Tue Feb 26, 2008 11:26 pm

Re: Limited Codecs in Mac OS

Post by Debo »

Well, the PNG codec doesn't really cut it.

Image

On the left is the original image. On the right is the exported movie with PNG codec. There's banding visible, which will become worse once I bring the file into FCE. This goes for all codecs except ones like Uncompressed or, long ago, "None". But that's not an option anymore.

This banding is not visible during the preview. Only afterwards.
User avatar
slowtiger
Posts: 6067
Joined: Thu Feb 16, 2006 6:53 pm
Location: Berlin, Germany
Contact:

Re: Limited Codecs in Mac OS

Post by slowtiger »

Sure you chose the "millions of colours" option within PNG codec?
Debo
Posts: 63
Joined: Tue Feb 26, 2008 11:26 pm

Re: Limited Codecs in Mac OS

Post by Debo »

As far as I can see, that's not a selectable option.

Export Animation... > QuickTime Movie > OK > Format: PNG, Quality: Maximum, v Include Alpha Channel.
User avatar
slowtiger
Posts: 6067
Joined: Thu Feb 16, 2006 6:53 pm
Location: Berlin, Germany
Contact:

Re: Limited Codecs in Mac OS

Post by slowtiger »

Settings seem OK. But you need to understand RGB limitations in gradients.

RGB means a channels of 8bit for each of the three colours. This means each channel can only have 256 different levels of gray.

Imagine (or create in Gimp or Photoshop) an image of 256 x 256 px. Create a gradient from black to white vertically over the complete height. You will get an image with 256 lines, each of a different gray level or shade. If you enlarge the view, you'll clearly see the lines.

Now do the same gradient over an image of 2560 x 2560 px. You'll get lines of exactly 10 px height now. Within these 10 px there is no further gradient because there is no number between integers 127 and 128 (this would be the center of the image). Memorize: if the distance in px is larger than the distance in tone value/shade, you'll get bands.

Now have a look at your colours. There's not much distance in the values. You have a gradient from RGB 14 18 43 to RGB 73 36 18. The distances for each channel are R 59, G 18, and B 25. So the blue channel has to spread only 25 values over a distance of 273 px (your example pic, but the real production frame will be much biger), which means a band of at least 10 px width for each value. Of course bands of that size are visible! And you can't avoid them, since there's no other number between two integers. (The bands in the green channel are even 15 px wide.)

So what can you do? Graphic programs, when creating a gradient, just distribute the available values evenly. Some are able to apply dithering, which means that instead of a defined border the edges of the bands get stirred up pixelwise, thus making them less visible. I did this via exporting to PNG in my example, forcing Photoshop to interpret the gradient in just 16 colours. As you see in the contrast enhanced portion the bands are now invisible.

Image (download and enlarge view to study in detail)

AS doesn't offer much control over this. But you always can add textures to your shapes, thus camouflaging the bands effectively. I try to avoid flat colours and gradients anyway. It's a matter of choosing the right style for the available format, in this case RGB. (That's why big 3D packages use 10-bit RGB instead of 8-bit - it gives them a whopping 1,024 shades range!)

So it's not the PNG codec's fault.
Debo
Posts: 63
Joined: Tue Feb 26, 2008 11:26 pm

Re: Limited Codecs in Mac OS

Post by Debo »

Apart from using .PNG files in Photoshop, I always apply some noise to an image to counter the worst banding. So that's two counter-measures. However, it still appears. I could at least reduce it to a minimum when using no codec or Uncompressed, the latter of which is still in Final Cut Express, i.e. installed on my machine; the first of which requires no installation at all--you'd think.

I sometimes suspect the actual culprit is Final Cut Express. No matter how beautiful the clip you import is, when it comes out, the quality always takes a hit.

Where is the option to choose millions of colors for the PNG codec, then? Or is that not an option at all?
User avatar
slowtiger
Posts: 6067
Joined: Thu Feb 16, 2006 6:53 pm
Location: Berlin, Germany
Contact:

Re: Limited Codecs in Mac OS

Post by slowtiger »

Again, please: when you apply a gradient with a very small colour range over a large pixel distance in 8bit RGB, banding will inevitably occur. That's no matter of software or codec. Your only choice is, as you correctly state, to hide it with textures, or choose designs without flat colour gradients.

I think the "maximum" setting on your operating system equals the "millions of colours" setting on my Mac.
Debo
Posts: 63
Joined: Tue Feb 26, 2008 11:26 pm

Re: Limited Codecs in Mac OS

Post by Debo »

Yes, but the thing is: I know some banding is ultimately inevitable. That is why I already cover it up as much as possible. But before, I could reduce it to a minimum by either using no codec, or the Uncompressed codec. The loss of these is what this thread is about. If these codecs are provided by my operation system, then I don't understand I can still use them in Final Cut Express. Or why, in AS, I don't have the option to not use any codec at all ("None"), because certainly that requires no installation of any codec anyway.

I'm upgrading from 5.6 to 9.1, I have the newest OS, I just don't accept that the quality takes such a hit when it's not necessary.
massypehel
Posts: 3
Joined: Sat Oct 06, 2012 10:36 am

Re: Limited Codecs in Mac OS

Post by massypehel »

This is very essential for me to know these useful guideline. Thanks for sharing here.

I got there is source such type of news publish here: Signs ft lauderdale
Last edited by massypehel on Wed Dec 26, 2012 7:07 am, edited 2 times in total.
User avatar
slowtiger
Posts: 6067
Joined: Thu Feb 16, 2006 6:53 pm
Location: Berlin, Germany
Contact:

Re: Limited Codecs in Mac OS

Post by slowtiger »

Repeat: It's not the codec. PNG saves whatever AS renders, no changes, no loss of quality. Deal with it, or get an application which uses 10bit RGB.
Debo
Posts: 63
Joined: Tue Feb 26, 2008 11:26 pm

Re: Limited Codecs in Mac OS

Post by Debo »

Hey. Cool down. I'm not saying it is the codec. I'm trying to figure this out, I'm speculating, I even said the culprit might be FCE. Because anyway you look at it, there's obvious loss of quality all of a sudden in my new workflow, now I use 9.1.

With all respect, based on a previous conversation here: viewtopic.php?f=5&t=16114, I don't believe you'd even be able to see this quality loss. You say you work on a Mac, but it must be an older one as your settings give you different options ("Millions of Colors" for example). Similar thing here: viewtopic.php?f=5&t=19459.

I'm sure you mean well, and I appreciate your effort, but you don't seem to be able to help me here.
Post Reply