What are physical causes of clipping?

Question

What exactly limits modern digital camera sensors in capturing light intensity beyond certain point?

Answer 1

What exactly limits modern digital camera sensors in capturing light intensity beyond certain point?

In terms of the physical properties of the sensor itself:

The number of photon strikes and the number of free electrons resulting from such photon strikes until there are no more available electrons with the potential to be freed within each photosite (a/k/a sensel, pixel well, etc.) define its full well capacity. It's not much different from film, in which full saturation is reached when there are no remaining silver halide crystals in the emulsion that don't already have enough 'sensitivity specks' to be transformed into atomic silver by developer. The main difference is the shape of the response curves when each technology approaches full capacity. Digital results in the same number of electrons per photon¹ being released until full well capacity is reached. As film nears full saturation, more and more light energy (or development time) is needed to affect the remaining silver salts.

In terms of recording the analog voltages as digital data:

When the analog voltage from each photosite (a/k/a 'sensel', 'pixel well', etc.) is read from the sensor, amplification is applied to the signal. The camera's ISO setting determines how much amplification is applied. For each stop increase of ISO, twice as much amplification is applied. If the camera's "base" sensitivity (for simplicity's sake, let's call ISO 100 an amplification of 1.00X in which input voltage equals output voltage) is used, then photosites that reached full well capacity should result in a maximum voltage reading on the post amplification analog circuit feeding the ADC. If ISO 200 (2.0X amplification) is used, the voltage from any sensel that reached one-half (1/2) full well capacity or more is amplified to the maximum voltage allowed on the post amplification circuit. ISO 400 (4X amplification) results in any sensel that reached one-quarter (1/4) full well capacity or more being recorded at the maximum value, and so on.

Any amplification greater than 1.0X will apply a "ceiling" lower than the full well capacity of each photosite. When high amplification is used, signals weaker than full well capacity also reach the maximum voltage capacity of the circuits downstream from the amplifier. Any pre-amplified signal level that is strong enough to "peg the meter" after amplification is indistinguishable from any other pre-amplified signal level that will also "peg the meter."

When these amplified analog signals are converted to digital data by the analog-to-digital convertor (ADC), signals at the maximum voltage capacity of the circuit are assigned the maximum value allowed by the bit depth of the analog-to-digital conversion. If converted to 8-bit values, voltages are assigned a value in binary between 0-255. The maximum signal allowed by the analog circuit feeding the ADC would be recorded as 255. If 14-bit, voltages are assigned a value between 0-16,383 with the maximum value assigned a binary value of 16,383, and so on.

The takeaway for when you are actually taking pictures:

You'll get the most difference and the finest number of gradations between the brightest and darkest² elements in the scene you are photographing when the amplification is at the camera's "base" sensitivity and the shutter time and aperture are combined to give the brightest elements in the scene just enough exposure to be at or near full saturation. Using a higher ISO value is useful if it is not possible to expose for that long or with a wide enough aperture to approach full saturation of the highlights in the scene for the image you wish to make. But using a higher ISO comes at a price. The total dynamic range is reduced by the higher amplification of the electrical signals coming off the sensor.

So why don't we always shoot at ISO 100, or whatever the camera's base ISO is, and then push the exposure later in post? Because doing it that way tends to amplify "noise" in the image even more than shooting at higher ISO values does. How much more depends on how much and where noise reduction is done to the signal. But reducing the influence of noise by applying noise reduction to the analog voltages coming off the sensor also comes with a price - very dim point sources of light are often filtered out as "noise". That's why some cameras with very good low light/high ISO performance, in terms of noise reduction, are also known as "star eaters" by astrophotographers.

_{¹ There is a slight variation in the energy contained in a photon based on the frequency at which it is oscillating. Photons oscillating at lower frequencies release slightly less energy when striking the sensel than photons oscillating at higher frequencies. But for photons oscillating at a specific frequency/wavelength, the amount of energy released when striking the bottom of a pixel well is the same until full well capacity is reached.}

_{² We call the difference between the darkest and brightest elements that can be recorded by a sensor (or film) the dynamic range of the recording medium. For each stop of increase in sensitivity (ISO) with a digital camera, the linear voltage difference between "zero" and "full saturation" is halved. When converted to logarithmic scales, such as 'Ev', doubling the sensitivity results in a reduction of one 'stop' of dynamic range (all else being equal, which it rarely ever is).}

Answer 2

Adding to Michael Clark's excellent answer (describing full-well capacity clipping and ADC clipping), there are several other points in a digital photography pipeline where clipping can occur:

• For non-RAW images, during on-device color correction/automatic gamma adjustment prior to compression, and during the compression itself.

  When you compress an image as JPEG or MPEG, the hardware truncates the bit depth to whatever the compressed medium supports, which is typically much less than the hardware bit depth. Because of that truncation, values near both brightness extremes get lost.

  Prior to compression, your camera applies color correction and gamma adjustments that can affect the effective dynamic range that fits within the limited bit depth provided by the compressor. For example, when recording video in Canon Log mode, the darkest and lightest parts of the scene are mathematically pulled towards the center so that the effective dynamic range increases significantly, and fewer parts of the image will be clipped on either end of the range.

• During post-processing. When performing post-processing that significantly alters the brightness of an image, it is possible for early stages of the computation to actually cause values to exceed the range that can be correctly represented by the number of bits used to hold them. While rare, this sometimes does occur, and when it does, it can cause clipping even in areas of the photo that are not actually clipped in the original image.

• During color gamut correction while printing or displaying the image. When performing color correction, sometimes you can get values that fall outside the gamut that can be accurately reproduced by the output medium. At that point, the color engine has to decide what to do with those out-of-gamut values. This also effectively results in clipping, though it visually looks somewhat different from what most people think of when they talk about clipping, usually resulting in things looking the wrong color.

Answer 3

The easy empirical explanation:

Look at a very bright bulb, if the light is bright enough you won’t be able to see the inside of the bulb because your pupils can close more and there is still too much light striking your retina, saturating it and the information that reach your brain is clipped (you only see bright light but not the details within the light). That is one of the reason why if you try, you should not do it, to look at directly at a clear sky midday sun you won’t be able to see the sun but an intense light (Beware that trying to do it without the proper protection can actually permanently harm your eyes or your photographic equipment, lenses and sensor)

Any sensor behaves the same way (from your camera or otherwise). Once the signal (in this case light) is too high for its capacity (reaches the saturation level), it will clip any additional information, it’s unable to discern more signal, passing on just a flat high signal without any valuable information in it.