Pixels and photodiodes

Question

A photodiode is the smallest element of a digital sensor, while a pixel is the smallest piece of information in a digital image. I wonder if the two terms can be used interchangeably or there are some steps in the middle before a photodiode becomes a pixel.

Answer 1

The pixels you get in a finished, camera-independent image file - a JPEG, TIFF, BMP, ...) file you get either from your camera or from your RAW processor software - are usually the results of multiple photodiode outputs (sensor pixels) combined, due to the way color filters arrays work.

The pixels in an image file, or in an image displayed on a monitor, are RGB pixels which each have 3 color channels to themselves, whereas a sensor pixel from a color digital camera will represent only one color channel - processing in camera or computer will interpolate the other color channels from adjacent sensor pixels following a so called demosaicing algorithm. You do get the same amount of output pixels as you have sensor pixels - the luminance (brightness) resolution of the camera will still be equivalent (unless limited by an AA filter, see below) to the nominal resolution of the sensor, whereas the color resolution is actually somewhat lower.

This does not affect the output as much as one would assume as long as one does not photograph extremely fine tartan-like patterns of contrasting colors, or subjects made of pure color noise. This could result in so called moire effects or other color artifacts.

The actual resolution of many sensors is actually optically limited by so-called antialiasing (AA) filters. This is because even a pure luminance sensor (black&white) can actually only resolve as high as its nominal resolution under optimal and very unreliable conditions: If you projected a checkerboard pattern with fields equivalent in size to sensor pixels on to a sensor, you could end up with a perfect reproduction of the pattern... or with a perfect uniform gray, if the sensor was misaligned 1/2 pixel wide with the checkerboard pattern. Conservatively (following Nyquist's sampling theorem) you would have to cut the resolution by half in each dimension (making a 6MP sensor of a 24MP sensor); in practice, compromises are made (The filter will attenuate contrast progressively the closer you get to theoretical resolution),or the AA filter is omitted, either leaving it to software to recognize such artifacts and correct them, or simply accepting the risk.

Answer 2

The basics of how a photosite is constructed are covered here: What is the structure of a photosite? — each photodiode has associated other electronics, and also (usually) a color filter (as well as other filters in a stack).

In order to produce a full-color image, information from each photosite is combined in a process called "demosaicing". Take a look at What does an unprocessed RAW file look like?, which shows image files constructed from each photosite directly in various states of processing.

In a "naïve" conversion, each photosite maps 1:1 to a pixel in the output image, although the actual data for each pixel is influenced by the surrounding pixels because of the demosaicing process. But, these days, it's typical to also convert for lens distortion, chromatic aberration, and similar, and those mappings stretch and move data around, such that there's not really a one-to-one correspondence between photosites and final image pixels. (It's still roughly true that the number of photosites is about the same as the number of pixels in the final image.)

Answer 3

The terms photodiode, photosite, and pixel are frequently used interchangeably; and of the three, pixel(s) is the most common. I.e. I have never heard/seen a camera referred to as having 36 million photodiodes.

Answer 4

A lot of folks do use the term interchangeably. That's regrettable, though, because there are fundamental differences.

To start with, each photosite (a/k/a sensel or pixel well), only records a single brightness value, not three different values for a Red, Green, and Blue color channel. It is the case that each photosite has a color filter in front of it. However, none of them record only red, only green, or only blue light. There's overlap in what wavelengths of light get through each color filter of the Bayer mask. This mimics the way the three types of cones in our retinas respond to different wavelengths of light.

For a much more detailed discussion of this, please see: RAW files store 3 colors per pixel, or only one?

Secondly, the filters in front of each photosite on our camera's sensor are not the same exact colors as the three colors we use in our RGB reproduction systems. The "blue" and "green" filters are fairly close to the colors we call Blue and Green in the RGB system, but the "red" filters are nowhere near what we call Red in the RGB system. To get RGB values for each pixel in an image, we must interpolate colors from the monochromatic luminance values measured by photosites that are behind colored filters that do not correspond to the colors we use for 'RGB' reproduction systems. We do this interpolation using information from surrounding photosites as well as the photosite in question. We call this process demosaicing. This is very similar to the way our brains perceive color based on various wavelengths or combinations of wavelengths of light.

For a much more detailed look at this, please see: Why are Red, Green, and Blue the primary colors of light?

This answer to Why don't mainstream sensors use CYM filters instead of RGB? illustrates how the colors used in a Bayer mask differ from the colors we use for 'RGB'. The curves are measured responses of the "red", "green", and "blue" filtered photosites to light at various wavelengths by a specific camera sensor. The vertical lines are the approximate positions of "Red", "Green", and "Blue" used by RGB color reproductions systems. I've also included "Yellow" to show that the peak response of the "red" filtered photosites is much closer to what we call "Yellow" than to what we call "Red."

Notice that the peak response of the "blue" filtered photosites is around 455nm, while the "Blue" we use in 'RGB' is about 480nm. The peak response of the "green" filtered photosites is around 545nm, while the "Green" we use in 'RGB' is about 230nm. Most notably, the peak response of "red" filtered photosites is around 595nm, while the "Red" we use in 'RGB' is about 640nm. The "red" filters on our cameras' Bayer masks are more yellow-orange than they are "red"!

Finally, the monochromatic brightness values for each photosite from a camera's sensor are linear. That is, the response to light from total black to total white is a straight slope. Our human vision systems, as well as the reproduction systems that we use to display images, have a logarithmic response with an 'S-shaped' curve at the very darkest and brightest ends. The linear brightness values from a camera's sensor need to be converted to response curves that mimic how our brains process the information gathered by our retinas. We call this gamma correction. It is similar, but not the same, as the gamma correction that our graphics adapters use when sending an RGB signal to a monitor.

Here is a raw file that has been demosaiced and rendered linearly (no curve). It's a fairly extreme example of a scene with a very wide dynamic range (the difference between the brightest and darkest parts of the scene).

_{The reason the gamma correction line is curved (in the shape of a near perfect curve for y=(√2)^x when x is between -10 and +4) in the histogram is because the exposure stop scale is exponential - there is really twice as much distance between each set of two stops as you move to the right as there was between the previous two stops. If the exposure scale were rendered that way, then the response "curve" you see would be a straight diagonal line.}

Here is the same RAW file rendered with a fairly standard set of gamma curves: the neutral picture style and no brightness adjustment.

Here it is when rendered with some highly customized light curves and +1.17 stops brightness adjustment to raise the shadows and then reign the highlights back in a little.

Here is the final edited image after some additional, fairly aggressive tone mapping has been applied using Canon's DPP HDR module to the single RAW file as originally edited with the customized light curves.

Put all of this together, and it is easy to see how the information contained in a raw image file must be processed before it resembles anything we would consider a viewable image. For more about what that information "looks" like at various stages in the conversion process from a raw file to an image, please see: What does an unprocessed RAW file look like?