How does on-sensor phase detection AF not impact the image?

Question

My understanding is that some of the photosites on the sensor have additional functionality in them to measure phase, which in turn uses some of the energy that would have otherwise registered as luminosity and eventually pixel brightness. Whose job is it to compensate for this when reading back the pixel values? The sensor circuitry? The camera CPU or DSP? The RAW demosaicing code?

Answer 1

How does on-sensor phase detection AF not impact the image?

It does. But the effect is usually so miniscule as to not be noticeable. With early "hybrid" sensors, only a few thousand of the millions of photosites on the imaging sensor were used in this way. If a sensor contains 20 million photosites (pixels), and 20,000 of them are used for sensor based phase detection, that amounts to 0.1% of the sensor's total number of photosites. Let's say the affected 20,000 pixels are only 80% as efficient as the other 'regular' pixels. Losing 20% of the signal from 0.1% of the total number of pixels amounts to losing 0.02% of the sensor's total signal compared to a similar sensor in which all of the photosites were the 'normal' ones instead of the 'PD' ones.

With more recent sensor designs almost all of the photosites on the sensor can be 'dual well' photosites. These designs do result in more measurably reduced efficiency. But the loss is considered acceptable in exchange for the increased AF speed and functionality they provide.

I'm more interested in the local effects - based on what you're describing it should be in principle possible to take a photo of a (very uniformly lit) gray card, apply some radical curve transformation on the result and clearly see the exact locations of all the PDAF pixels - if they are sporadically spread they would be filtered away by simplest noise reduction but if closely clustered than they could basically be brought out as darkened spots...

In the first case (a few modified photo sites), they're usually fairly evenly distributed over most of the camera's field of view. There's nowhere where they are so concentrated to have any perceivable impact.

In the second case (most of the sensor's photosites), the impact is fairly equal everywhere.

...unless somewhere along the readout process the sensors/camera corrects for them by compensating the values read for the known loss.

This type of processing is already fairly common for many digital cameras.

• Sensors are calibrated to account for differences between individual photosites before they leave the factory.
• 'Stuck' and 'Dead' pixels are mapped out.
• Due to the reduced response of digital sensors to light from angles other than perpendicular, the internal image processing pipeline for many cameras already compensates for edge/corner brightness based on the specific lens connected to the camera. This is particularly the case with very wide angle lenses. The sensor/camera profiles used by external raw processing applications also commonly do this. This is in addition to any 'peripheral light falloff' correction done to correct what is commonly called 'vignetting' due to light falloff on the edges of the image circle projected by the lens.

Answer 2

As suspected it does in fact and in a way that can be visible under exposure compensation:

https://www.dpreview.com/reviews/nikon-z7-first-impressions-review/ (search for "PDAF Banding and Striping")

And PDAF sites can definitely be visualized:

http://photonstophotos.net/Charts/Sensor_Heatmaps.htm#mode=39,camera=Nikon%20Z%207,suffix=14

Given this I guess the question should be reframed as "Is it possible for cameras or raw converters to compensate for PDAF sites ?"