Does super-sampling produce good results?

Question

The Lumia 1020 has a 41MP sensor in a PHONE. It down-samples the photo to 5 megapixels, improving the photo quality. But would you get a better photo by using a native 5MP sensor of the same size?

In general, given two sensors of the same size, but with one of them having a higher resolution than the other, would the result of downsampling the higher resolution image to the lower resolution be better or worse than the image obtained from the lower resolution sensor?

(You can assume that a high-quality image downsampling algorithm is being used. And yes, a higher resolution photo gives you flexibility to crop, but for the purpose of this question, let's assume we are not going to crop the photo later.)

Answer 1

The pixel reduction from 41 to 8mp has the impact of improving the accuracy of colour capture and reducing the appearance of sensor noise since for each output pixel you've got a handful of pixels to calculate the best value. Obviously there's a tradeoff that you've got less spatial resolution and it comes down to what makes for the most pleasing image in the end.

Digital supersampling systems aren't new. Fuji's Super CCD cameras (like the 2002 S2 Pro which sampled 12megapixels but output at 6) were widely applauded for the quality of their colour reproduction/tonal range compared to their Nikon & Canon peers.

According to reviews the output between the likes of the Lumia 1020 and iPhone5 tend to be comparable under good lighting. But PureView devices generate better output in the kind of challenging situations where people are less likely to be carrying a camera with a larger sensor. In practice the whole system performs as well or better than a native 8mp sensor in a similar package.

The maths & physics involved mean that beyond a certain pixel density there are diminishing returns. Probably Nokia are very much at this point but the scope for improvement might be more about their processing than goosing the pixel density much further.

Answer 2

I think that the answer could be "yes, if well done". The best advantage I see is the possibility to have a digital antialiasing filter much more sharper than a physical one.

Let me explain (geeky here (1)) in one dimension. If you want to sample something at, say, 100 points/mm, the Nyquist theorem says that for avoiding aliasing (typically shown as moire in the images) you should cut all frequencies in the input signal above 50 points/mm. Notice that once aliased data is in, is in. You can't distinguish it from the real data, so it's impossible to remove.

Now, making a physical filter that cuts completely the frequencies above 50 p/mm and let pass the frequencies below is impossible; the filters have to transition from "pass" to "no pass" in a finite range of frequencies. Filters that have faster transition(2) are much more complex to do (especially the optical ones). Let say that a reasonable "transition band" is 10 p/mm for a physical filter.

So you have to compromise between aliasing and band (moire and sharpness in imaging). for example, you can filter from 20 p/mm and have very little moire but a loss in sharpness; or filter at 45 p/mm and risk a bit of moire with more sharpness, or pass on the filter...

If you oversample at say 1000 p/mm you move simply the problem up, no? But suppose you really want 50 p/mm. So you now put a physical antialiasing filter at 200 p/mm (easy to do, no aliasing). And then you can use a digital filter before (re)sampling at 100p/mm --- and this filter is just software so that you can make it really much more fast at a reasonable price, especially with modern hardware: say you can do the transition in 1 p/mm. So you have at the end the equivalent of the first system, but you can put the final filter at 49 p/mm and having no moire whatsoever (3).

On the negative side, having more pixel means having more electronics, so it means that the total area of the chip used for sensing is smaller in a higher megapixel sensor. That could be in part corrected by the microlenses, but it generally means that a hit in noise (high ISO performance) is to be expected, too.

(1) my background is analog electronics. This post describes a technique that is widely used in digitizing signals like audio or biological ones. Forgive me if I use a quite strange terminology.

(2) in electronics we call this filters "sharp", not "fast". I used fast to not cross meaning with sharpness of the image.

(3) No idea if Sharp do all of that. Just guessing.