Would a demosaic algorithm for black and white be useful?

Question

Given that the main purpose of demosaicing is to recover colour as accurately as possible, would there be any advantage to a "black and white only" demosaic algorithm? That is, instead of first recovering the colour and then converting black and white, might it be better to convert the RAW file directly to black and white?

I'm particularly interested in the image quality (e.g. dynamic range and sharpness). On a related note, which common demosaicing algorithms are most amenable for black and white conversion?

Answer 1

There is no way to convert a RAW file directly to black and white without recovering the colour first, unless your converter takes only one of the R,G,B pixel sets to produce an image. This approach would result in a substantial loss of resolution.

In order to not lose resolution when converting to black and white you have to use all R G and B pixels, which implicitly means colour calculations must be performed, at which point you might as well use one of the advanced colour demosaicing algorithms, and then convert the result to black and white.

Answer 2

You need demosaic algorithm even if you convert an image to B&W.

A reason for that is quite simple - otherwise you'd get sub-pixel artifacts all over the place. You need to realize that image recorded by sensor is quite messy. Let's take a look at the sample from Wikipedia:

Now imagine we don't do any demosaicing and just convert RAW into grayscale:

Well... you see the black holes? Red pixels didn't register anything in the background.

Now, let's compare that with demosaiced image converted to the gray scale (on a left):

You basically lose detail, but also lose a lot of artifacts that make the image rather unbearable. Image bypassing demosaicing also loses a lot of contrast, because of how the B&W conversion is performed. Finally the shades of colours that are in-between primary colors might be represented in rather unexpected ways, while large surfaces of red and blue will be in 3/4 blank.

I know that it's a simplification, and you might aim into creating an algorithm that's simply: more efficient in RAW conversion to B&W, but my point is that:

You need computed colour image to generate correct shades of gray in B&W photograph.

The good way to do B&W photography is by removing colour filter array completely - like Leica did in Monochrom - not by changing the RAW conversion. Otherwise you either get artifacts, or false shades of gray, or drop in resolution or all of these.

Add to this a fact that RAW->Bayer->B&W conversion gives you by far more options to enhance and edit image, and you got pretty much excellent solution that only can be overthrown by dedicated sensor construction. That's why you don't see dedicated B&W RAW converters that wouldn't fall back into demosaicing somewhere in the process.

Answer 3

Machine vision cameras with bayer filters can give greyscale images directly but it does this by demosaicking, converting to YUV, and sending only the V channel (the ones I normally use at least). If they had a better way by bypassing this colour reconstruction I think they would, as they are constantly pushing framerates (the typical camera I use runs 100FPS for example).

If it were to ignore the colour based demosaicking it could half the resolution and weighted average each 2x2 quad, but if you want full resolution it is better to use the normal colour demosaicking algorithm which tries to preserve edges better. If we know we want greyscale we just get a monochrome camera from the start, slap on a colour filter if we look for a certain colour, as this setup is vastly superior in image quality, reducing the need for resolution oversampling, which in turn allows use of a fast low resolution sensor with larger pixels, which in turn gives an even better image.

Answer 4

The effect of the color filters over each pixel well in the Bayer layer are the same as shooting B&W film with color filters over the lens: it changes the relationship of the gray levels of various colors in the scene being photographed. To get an accurate luminance level for all colors in the scene the signals from each pixel must be demosaiced. As others have mentioned, a sensor with no Bayer layer would yield a monochrome image that need not be demosaiced. This should result in better image sharpness if the circle of confusion from the lens is equal to or less than the width of each pixel.

In practical terms, I've noticed several things converting RAW files to monochrome using Canon's Digital Photo Professional (DPP).

1. White Balance adjustment can effect a change in overall perceived luminance in the same way that contrast adjustment can. As such, it can be used to fine tune contrast.
2. White Balance will also have an affect on the relative luminosity of different colors in the scene. This can be used to fine tune application of the "Orange", "Yellow", "Red", etc. filter effects. Red seems to be the most affected by this and is much darker at 2500K than at 10000K. Surprisingly, at least to me, is that blue tones do not demonstrate the reverse.
3. Since for all practical purposes there is no chrominance noise in a B&W photo, it can be left at "0".
4. The unsharpen mask tool will give much more control over sharpness than the simpler "Sharpness" slider. Especially if you have a few "warm" or "hot" pixels in the image, you can increase overall sharpness without emphasizing them.

Below are two versions of the same exposure shot on a Canon 7D with an EF 70-200mm f/2.8L IS II lens and a Kenco C-AF 2X Teleplus Pro 300 teleconverter. The image was cropped to 1000X1000 pixels. The first was converted using the in camera settings shown below it. The second was edited with the settings shown in the screen shot. In addition to the RAW tab, a Luminance Noise Reduction setting of 2 was applied, as was a Chromatic Aberration value of 99.

Answer 5

I would propose an algorithm like so (presumes your target is white and has consistent colour temperature):

• Demosaic RAW Bayer to RGB
• Downsample colour to grayscale
• Create a LUT between raw bayer values and grayscale values (this would need to be performed once per colour plane RGGB or RGB)
• Use the LUT per colour filter to transform the RAW Bayer directly to Grayscale without any inter-pixel filtering

In theory this would approach the results of a true monochrome sensor.