Does the curves tool in Photoshop (or any software) do anti-aliasing?

Question

I'm not talking about drawing smooth curves here, but rather the curves tool used for adjusting color levels. An illustration will be worth a thousand words, here. This is an adjustment in Gimp:

Adjustment:

Histogram Before:

Histogram After:

Here, we see that the tool has done the simple mapping of the 8-bit channel along the curve, which produces a zebra pattern in the resulting output.

Does any software handle this in a more sophisticated way, using an anti-aliasing algorithm so the full range of output values is used?

Answer 1

An 8-bit image will most likely have its pixel values truncated. What you need to do is interpolate the missing low order bits before applying a colour transformation. There is no way to know what the missing values are, you would have to make certain assumptions, such as smoothness.

To answer your question I don't know of any image editor that offers this feature. Why not? Speed for one, the curves/levels command in most photo editors is fast enough to work in real time so you can see the effect on your image. Now you could have a separate function that operates when not previewing, but this would be a pretty high level feature, and you'd have to implement it for all operations that apply scaling to pixel values. Finally it's a problem that completely goes away if you have a camera which supports RAW and adopt a 16-bit workflow.

You can always do the interpolation yourself, e.g. by converting to 16-bit, resampling at a higher resolution, downsampling. An alternative scheme that will probably work just as well would be to randomize the lower order bits (by adding a small amount of noise). Or a combination of resampling and randomization.

Don't be too put off by the look of the histogram in your image the quantization error of that operation is unlikely to be noticeable if there is any noise in the image provided you don't have any areas with shallow colour gradients (however in cases with smooth transitions and little noise, interpolation would work very well).

Answer 2

Although the color information loss is clearly not avoidable in this kind of 8-bit transformation, you can have a better behaving image if you accept a bit of a loss in sharpness...

I do not know in other programs, but in darktable you can use t he module "dithering":

I tried the same trick on a small jpeg image, and this is the result: before:

and after:

Now the histogram is much better behaved; it can reduce color banding in some case.

Answer 3

From a theoretical point of view there is no "True" way of getting those values back. convolution kernels (like guassian smoothing, or an edge preserving kernel) and dithering will make assumptions to bring them back, rank filters may not (like median).

However, there is no way to "know" how those values should have been before the quantization, so it is a subjective judgement, which convolution based filter you prefer. You might need to use multiple filters and mask your image as to where you want them applied.

The best way to overcome this is by processing in 16bit mode , so that the "zebra shaped value space" get resampled when going to 8bits. This should eliminate most skipped values in 8 bit space, unless you added a lot of contrast/gain.

Answer 4

All adjustments you make will be destructive in this way.

a couple best practices for minimizing the contrast damage that can occur from over-adjustment.

1. Start off with a completely neutral or logarithmically processed image in 16 bit

2. Dont make any permanent levels, curves, hue/sat or brightness contrast adjustments. keep everything in layers.

3. DONT USE LEVELS! Curves are much better at gradating values.

4. flatten and save into 8 bit.

As for the algorithmic question, I'm not aware of anything that can reverse or "handle" this any better. Is this for scientific work?