3

Some smartphone cameras support pixel binning, to switch between high-res and low-light modes.

An example would be the S20 ultra, which has an 108MP sensor that can bin pixels on a 3x3 grid, leading to 12MP images. Marketing material specifies two different pixel sizes for this sensor: 0.8 and 2.4 microns.

But how does this work in practice? Are there trade-offs involved?

Is this more than just software, to bin the pixels together? If I took an image at 108MP, would it not be possible to calculate an identical 12MP image out of this data, just as if the image had been taken at 12MP in the first place? To be honest, I can hardly believe that 3x3 0.8 micron pixels are the same as one pixel of 2.4 microns.

1

But how does this work in practice? Are there trade-offs involved?

There are always trade-offs involved. In the case of quad-pixel and now nona-pixel sensors, the amount of color resolution is reduced from what one would expect to get from the full number of pixels.

For 48MP quad-pixel sensors that output 12MP when using binning, their color resolution is equal to about 27MP when using algorithms to rearrange the subpixels among the logical pixels. It interpolates from this (on the left) to this (on the right):

enter image description here

Each group of four same-colored pixels is virtually "stretched" to 1.5 their linear size.

If I've done the math correctly (using 1/1.5 cubed instead of 1/1.5 squared), a 108MP nona-pixel sensor that gives 12MP image when binned will yield color resolution equal to 32 MP when interpolating in the above method. Or maybe I should have used the square of 1/2 instead of the square of 1/1.5? In which case you'd be right back to 27MP effective color resolution.

This article from xda-developers.com explains the effect with quad-pixel sensors in more detail.

Before 2018, essentially every smartphone camera had a Bayer color filter array. The Huawei P20 Pro and the Huawei Mate 20 Pro were the first phones to use Quad Bayer sensors. Simply put, a Quad Bayer sensor has less color resolution than a sensor with a standard Bayer layout. On the IMX586, for example, the physical color filters on the camera sensor only have an effective resolution of 12MP. The ISP of such sensors is able to achieve a virtual 48MP Bayer result out of the sensor by re-arranging the subpixels among the logical pixels. It should be clear that this approach isn’t as good as using a standard Bayer filter. What is the specific difference? According to AnandTech, the 48MP IMX586 has closer to 27MP of spatial resolution as it’s only able to increase spatial resolution half-way.

The link in the above quote includes this about quad-pixel sensors:

This means that the physical colour filters on the camera sensor only have an effective resolution of 12MP. Sony’s sensor ISP is able to achieve a virtual 48MP bayer result out of the sensor by rearranging the subpixel-data among the logical pixels. It’s to be noted that this method would result in an effective spatial resolution increase of only half-way to 48MP, and actual results would be of clarity somewhere in the range of a true 27MP bayer sensor.

Samsung themselves hype their tetracell (quad-pixel) technology, along with other sensor innovations here. But Dieter Bohn at The Verge wasn't very impressed by the actual performance that 108 MP promises for the ability to zoom.

| improve this answer | |
  • If one takes a picture of a floor with e.g. red and blue stripes receding into the distance, how could a quad-pixel sensor avoid Moire patterns other than by ensuring that the image projected on the sensor was slightly blurred, reducing the effective resolution to that of a sensor with one pixel per dot? – supercat Apr 2 at 20:51
0

Exactly the same thing happens if you downsize an image... the information from the "excess pixels" is combined. That's why all FF sensors of a given technology/generation will have essentially the same low light performance for the same size output image; regardless of their pixel count.

| improve this answer | |
  • Except it's not exactly the same because the Bayer array is no longer R-G-R-G on one line and G-B-G-B on the next line. For quad-pixel sensors it's now R-R-G-G on the first two lines and G-G-B-B on the next two lines. For nona-pixel arrays it is R-R-R-G-G-G on three lines and G-G-G-B-B-B on the next three lines. – Michael C Mar 10 at 12:41
  • @MichaelC, "downsampling" is actually better because the optimal sampling rate for a Bayer array is 4 pixels (RGBG) per airy disk. – Steven Kersting Mar 10 at 15:01
  • Well, kind of. "R" in the CFAs is not the same as "R" in RGB. Nor are "G" and "B" in the CFAs the same colors as "G" and "B in RGB. So colors still have to be interpolated from the raw data. – Michael C Mar 11 at 1:25
  • I'm aware... RGB filtration is more accurately long/middle/short wavelength centric. The point being that if you have the pixels for downsampling you also have them in the raw file for more accurate color information/demosaicing. – Steven Kersting Mar 11 at 14:40
  • But you don't have them arranged in the proper pattern for spatial resolution as fine as you would with a conventional Bayer array of the same number of photosites. To do the same thing as 4 pixel downsampling (2x2) with a conventional Bayer CFA, one would need to do 16 pixel downsampling (4x4) with a quad-pixel sensor and 36 pixel downsampling (6x6) with a nona-pixel sensor. – Michael C Mar 11 at 23:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.