Combining multiple exposures

Question

I find it interesting that the idea of combining bracketed exposures to increase the recorded range of brightness has gone mainstream (known as HDR) to, the point where it is built into the camera's jpeg rendering. (Aside: now that cameras are powerful enough to do that, sensor latitude has improved so we don't need to overcome it)

But there are other ways of cobining exposures. I've heard of stacking to scan over a depth of field, with various programs available to aid in that.

What I'm interested in at the moment is improving low-light/high-ISO shots through multiple exposures.

I read the suggestion here of stacking them in Photoshop (alignment of such stacks is now built-in) and simply taking the median of each pixel. That gave useful improvement from a burst of normal exposed shots.

But it could be much fancier than just taking the median. Besides smarter combining of the above stack, another variation is to make each camera exposure much shorter than the desired exposure, and add them together afterwards. In effect, the long exposure has the sansor buckets measued at intervals rather than only once. Just adding gets me nowhere better, but that allows profiling the variance and removing noise, in principle better than just using the median.

Is there existing software for doing these things? Since the techniques have come down to us from astronomy, the dSLR astrophotography enthusiasts might have brought them to home PCs. Assuming such software is not too specific for that kind of image, is that available?

Another thing I noticed from having to align a burst of shots, even from a tripod, is that they are not perfectly in register. That can be used to advantage: in some places even the dumb median method improved antialiasing of a straight line due to the jitter. I recall (again, from this site) this being done on purpose by astrophotography where it was (confusingly to me) called dithering (it means something different in computer graphics) for a closely related improvement. With whole-pixel alignment the simple stacking can he hurt or helped in certain ways due to sub-pixel registration differences, but a smarter stacker could know about that to get even better results, to the point where it is beneficial. Are any image-stacking programs out there doing stuff like this?

Answer 1

If you have a reasonable statistical model of the noise sources then yes, you can do better than median filtering, but not by that much. It's much easier to boost performance by simply shooting more images.

With regards to exploiting the slight misalignment of images, this can be used to increase resolution, the technique is called super-resolution and there are programs to do this for you if you google the term.

Answer 2

There is software that does what you are looking for. Fundamentally, averaging is the most effective way to reduce noise and increase signal strength. When you have no other algorithmic capability to identify and reject outliers, a median is usually the best approach. However, median will usually not give you the best SNR in the end, as it is simply selecting a value that it thinks is best. Using a mean, combined with some kind of sigma rejection, will usually yeild better results with lower standard deviation, and higher SNR.

A rejection algorithm uses statistics and a configurable number of standard deviations to identify and eliminate (either by simply discarding, or by adjusting the value of) sigma outliers. By doing this, you can eliminate rogue pixel values that often crop up in astrophotography. These may be from meteor tracks, airplanes, even cosmic ray strikes on the sensor. Combined with something called dithering (slightly offsetting the stars in each sub frame), after star alignment, hot pixels will be randomly distrubuted, and can also be rejected by such an algorithm. Once pixels that are identified as statistical outliers are either removed or shifted to within range (usually clamped to the nearest non-outlier boundry value, or set to the median value of the entire stack), the averaging algorithm actually does its thing to give you a final integrated image.

There are two programs that I have used to process my astrophotography. The first, and the one most beginners start with as it is free, is DSS or Deep Sky Stacker. You can use a simple averaging stacking algorithm, but combined with the Kappa-Sigma rejection algorithm, you can get very good, clean results. DSS is very easy to use, and also supports calibration with darks, biases and flats, as well as star registration and alignment.

The other program that I use most frequently these days is PixInsight. This is not a free program, but if you do a lot of astrophotography, it is highly recommended. PixInsight is a fully featured astro editing program, and uses some of the most advanced algorithms around for image calibration, integration, and processing. PixInsight supports a variety of advanced averaging algorithms, including Winsorized Sigma Clipping. This algorithm, combined with the basic averaging algorithm, is highly configurable and probably the most effective means I've seen of identifying and rejecting (by clamping to the nearest non-rejected bounding value) sigma outliers.

Another side benefit of using PixInsight, if you gather enough sub frames, is that you can also drizzle to increase resolution if you were imaging undersampled. PixInsight's drizzle algorithm is one of the few that are on the market that can actually apply rejection to drizzled integrations, giving you very clean, very high resolution results.