Apples

Apples

by Garik

submit your photo


Picture of the Week Themes
Suggest and vote on themes

Please participate in Meta
and help us grow.

Tell me more ×
Photography Stack Exchange is a question and answer site for professional, enthusiast and amateur photographers. It's 100% free, no registration required.

I understand the purpose of the anti-aliasing (AA) filter is to prevent moire. When digital cameras first emerged an AA filter was necessary to creat enough blur to prevent moire patterns. At that time the power of in camera processors was very limited. But why is it still necessary to place an AA filter over the sensor in modern DSLR cameras? Couldn't this be accomplished just as easily by the algorithms applied when the output from the sensor is being demosaiced? It would seem that the current processing power available in-camera would allow this now much more than even a few years ago. Canon's current Digic 5+ processor has over 100 times the processing power of the Digic III processor, which dwarfs the power of the earliest digital cameras. Especially when shooting RAW files, couldn't the AA blurring be done in the post processing stage? Is this the basic premise of the Nikon D800E, even though it uses a second filter to counteract the first?

Update

The Nikon D7100 announced this week (02/20/2013) is a 24MP APS-C camera that will have a pixel pitch of under 4µm and no low pass Anti-Aliasing Filter.

share|improve this question
It is not. There are already DSLRs with have no anti-alias filter, including the Pentax K-5 IIs, Nikon D800E, plus mirrorless models like the Olympus PEN E-PM2 and all Fujis (X-E1, X-Pro1). Plus they even announced fixed lens cameras without AA filter (X20 and X100S). – Itai Feb 7 at 15:46
And all of those cameras show color moire at times. – Kendall Helmstetter Gelner Feb 8 at 19:34
2  
Indeed but so will other cameras. I suspect that an anti-alias filter which avoids all moire would be too strong, so manufacturers use AA filters of lesser strength. As an example, in my K-5 IIs and K-5 II comparison, moire occurs on both camera, only much more with the K-5 IIs. – Itai Feb 10 at 1:58
1  
IIRC the new Nikon D7100 doesn't have one either. – James Snell Feb 22 at 11:11

3 Answers

up vote 5 down vote accepted

Aliasing is the result of repeating patterns of roughly the same frequency interfering with each other in an undesirable manner. In the case of photography, the higher frequencies of the image projected by the lens onto the sensor creates and interference pattern (moiré in this case) with the pixel grid. This interference only occurs when those frequencies are roughly the same, or when the sampling frequency of the sensor matches the wavelet frequency of the image. That is the Nyquist limit. Note...that is an analog issue...moiré occurs because of interference that occurs real-time in the real-world before the image is actually exposed.

Once the image is exposed, that interference pattern is effectively "baked in". You can use software to some degree to clean moiré patterns up in post, but it is minimally effective when compared to a physical low pass (AA) filter in front of the sensor. The loss in detail due to moiré can also be greater than that lost to an AA filter, as moiré is effectively nonsense data, where slightly blurred detail could still be useful.

An AA filter is just designed to blur those frequencies at Nyquist so they do not create any interference patterns. The reason we still need AA filters is because image sensors and lenses are still capable of resolving down to the same frequency. When sensors improve to the point where the sampling frequency of the sensor itself is consistently higher than even the best lenses at their optimal aperture, then the need for an AA filter would diminish. The lens itself would effectively handle the necessary blurring for us, and interference patterns would never emerge in the first place.

share|improve this answer
Here is part a comment posted to photo.stackexchange.com/questions/10755/…. Do you still believe it to be accurate? If so, how is the pattern baked in until the RAW data has been demosaiced? "Ironically, at least with RAW, the theoretical nyquist limit does not always seem to be a hard limit, which is probably due to the different wavelengths of red, green, and blue light and the distribution of RGB pixels in a sensor. – jrista♦ Apr 10 '11 at 18:50" – Michael Clark Feb 7 at 13:30
1  
I believe I was talking about resolution in general there, and not directly to aliasing in the recorded digital signal. The nyquist limit is kind of a tough thing to nail down in a bayer sensor due to the uneven pattern of RGRG and GBGB rows. The spatial resolution of green is higher than the spatial resolution of either red or blue, so the nyquist limit in red or blue light is at a lower frequency than the nyquist limit in green light. The nyquist limit in a demosaiced image is kind of tough to call exactly, so it becomes a bit of a fuzzy band, rather than a concrete mathematical limit. – jrista Feb 9 at 4:25
1  
...that pattern becomes part of the image. Even if you knew the exact wavelet characteristics of the virtual image, and could produce a fourier series of then, you would have to change the orientation of the image relative to the virtual concept of the sensor to eliminate the moire "perfectly". That is a lot of excessively intense, highly mathematical work...assuming you knew the EXACT nature of the original virtual image signal and its relation to the sensor. Once aliasing is baked into a RAW, it is pretty much done, there really isn't any undoing it without softening detail. – jrista Feb 9 at 4:30
1  
I know all about the difference in frequency between red/blue and green. As for all current optical AA filters only filtering at nyquist, it really depends on the camera. Not all AA filters are designed exactly the same, and even for the same brand, different models and different lines often have AA filters that behave differently. I know that historically the 1D and 5D lines have allowed SOME frequencies just above nyquist through, however I think its a matter of balancing with lens resolution. – jrista Feb 10 at 17:12
1  
On sensors with smaller pixels, such as the Canon 18mp APS-C, the D800, the D3200, pixels are getting really, really small. Outside of a small segment if really new lenses (like Canon's Mark II L-series generation, and then, only the ones released within the last two-three years) can resolve enough detail to significantly outresolve the sensor and cause aliasing at frequencies higher than nyquist. Filter at around nyquist, and the lens itself will blur detail beyond that. I think that is part of the reason the 5D line has had an overly strong AA filter...lenses outresolve it more easily. – jrista Feb 10 at 17:15
show 7 more comments

You can't get the same effect in software. You can get somewhere nearby, given certain assumptions. But the AA filter spreads light so that it strikes multiple different coloured pixels giving you information that is absent from the no-AA filter sensor.

The Nikon D800E doesn't do anything at all to try and replicate the AA filter. If there are high frequency patterns in the image, you get moire and that's your problem - you have to deal with it!

Aliasing is at it's worse when the frequency of detail in the image is very close to the sampling frequency. For older cameras with low resolution sensors (and hence low frequency sampling) moire was a serious problem with lots of types of image detail so AA filters were strong (nothing to do with limited processing power). Now we have much higher sampling frequencies, it takes much higher frequency image details for moire to show up.

Eventually sampling frequencies will be so high the necessary high frequency object details wont make it past lens aberrations and diffraction effects, making the AA filter redundant. This is partly the reason that some MF backs lack an AA filter, super high resolution plus fashion photographers who like to shoot at f/32 with giant Profoto power packs proving lighting.

share|improve this answer
It seems to me the interpolation done in the demosaicing process could be modified to accomplish the exact same thing, since averaging adjacent pixels is what is done there. The Nikon D800E has two AA filter components just like other cameras, but instead of one polarizing light horizontally and the other polarizing it vertically the second is 180 degrees from the first and takes the polarized rays from the first and combines them back into one stream. See photo.stackexchange.com/questions/22720/… – Michael Clark Feb 7 at 12:37
2  
@MichaelClark No you can't get the same effect in the demosaicing process. A single point of light hitting the D800E sensor will only produce charge at one photosite. There is no way to tell what colour that light was by looking at neighbouring pixels, the information has been lost forever. The same point of light hitting the D800 sensor (with AA filter) will hit one pixel strongly and the surrounding pixels to a lesser extent. As the neighbouring pixels have different colour filters by looking at their intensities it is possible for a demosaicing algorithm to estimate the colour of the light. – Matt Grum Feb 7 at 12:53
1  
@MichaelClark The only reason the D800E has this arrangement is to simplify the manufacturing process, it's much easier to change the orientation of one of the filters at the input stage than it would be to swap two filters for a clear glass element - ultimately the filter stack has to be the same height as it has a refractive effect and modern lens designs take this into account. Simply not putting either filter on the D800E would introduce a subtle aberration in the images. – Matt Grum Feb 7 at 13:03
But at the same time that single point of light is hitting one sensor site, corresponding points of light are hitting all of the adjacent sensor sites and the AA filter is causing all of them to spill light onto each other. Don't most demosaicing algorithms use interpolation to compare the luminosity levels of not only the immediate pixel wells but also other near pixels wells with the same color sensitivity? Effectively, isn't blurring adjacent pixels into each other mathematically what you are doing? – Michael Clark Feb 7 at 13:22

The physics simply doesn't work that way. Aliasing irreversibly transforms frequencies past the Nyquist limit to appear as frequencies below the limit, although those "aliases" aren't really there. No amount of processing a aliased signal can recover the original signal in the general case. The fancy mathematical explanations are rather long to get into unless you've had a class in sampling theory and digital signal processing. If you had though, you wouldn't be asking the question. Unfortunately then the best answer is simply "That't not how the physics works. Sorry, but you're going to have to trust me on this.".

To try to give some rough feel that the above might be true, consider the case of a picture of a brick wall. Without a AA filter, there will be moire patterns (which are actually the aliases) making the brick lines look wavy. You have never seen the real building, only the picture with the wavy lines.

How do you know the real bricks weren't laid down in a wavy pattern? You assume they weren't from your general knowledge of bricks and human experience of seeing brick walls. However, could someone just to make a point deliberately make brick wall so that it looked in real life (when viewed with your own eyes) like the picture? Yes they could. Therefore, is it possible to mathematically distinguish a aliased picture of a normal brick wall and a faithful picture of a deliberately wavy brick wall? No, it is not. In fact you can't really tell the difference either, except that your intution about what a picture probably represents may give you the impression that you can. Again, strictly speaking you can't tell whether the wavies are moire pattern artifacts or are real.

Software can't magically remove the wavies because it doesn't know what is real and what is not. Mathematically it can be shown that it can't know, at least by only looking at the wavy image.

A brick wall may be a obvious case where you can know that the aliased picture is wrong, but there are many more subtle cases where you really don't know, and may not even be aware that aliasing is going on.

Added in response to comments:

The difference between aliasing a audio signal and a image is only that the former is 1D and the latter 2D. The theory and any math to realize effects is still the same, just that it is applied in 2D when dealing with images. If the samples are on a regular rectangular grid, like they are in a digital camera, then some other interesting issues come up. For example, the sample frequency is sqrt(2) lower (about 1.4x lower) along the diagonal directions as apposed to the axis-aligned directions. However, sampling theory, Nyquist rate, and what aliases really are is not different in a 2D signal than in a 1D signal. The main difference seems to be that this can be harder for those not used to thinking in frequency space to wrap their mind around and project what it all means in terms of what you see in a picture.

Again, no you can't "demosaic" a signal after the fact, at least not in the general case where you don't know what the original is supposed to be. Moire patterns caused by sampling a continuous image are aliases. The same math applies to them just as it applies to high frequencies aliasing into a audio stream and sounding like background whistles. It's the same stuff, with the same theory to explain it, and the same solution to deal with it.

That solution is to eliminate the frequencies above the Nyquist limit before sampling. In audio that can be done with a simple low pass filter you could possibly make from a resistor and capacitor. In image sampling, you still need a low pass filter, in this case it's taking some of the light that would hit only a single pixel and spreading it out to neighboring pixels. Visually, this looks like a slight blurring of the image before it is sampled. High frequency content looks like fine detail or sharp edges in a picture. Conversely, sharp edges and fine detail contain high frequencies. It is exactly these high frequencies that get converted to aliases in the sampled image. Some aliases are what we call moire patterns when the original had some regular content. Some aliases give the "stair step" effect to lines or edges, especially when they are nearly vertical or horizontal. There are other visual effects caused by aliases.

Just because the independent axis in audio signals is time and the independent axes (two of them since the signal is 2D) of a image are distance doesn't invalidate the math or somehow make it different between audio signals and images. Probably because the theory and applications of aliasing and anti-aliasing were developed on 1D signals that were time-based voltages, the term "time domain" is used to contrast to "frequency domain". In a image, the non-frequency space representation is technically the "distance domain", but for simplicity in signal processing it is often referred to as the "time domain" nonetheless. Don't let that distract you from what aliasing really is. And no, it's not evidence at all that the theory doesn't apply to images, only that a misleading choice of words is sometimes used to describe things due to historical reasons. In fact, the shortcut "time domain" being applied to the not-frequency domain of images is actually because the theory is the same between images and true time-based signals. Aliasing is aliasing regardless of what the independent axis (or axes) happen to be.

Unless you are willing to delve into this at the level of a couple college courses on sampling theory and signal processing, in the end you're just going to have to trust those that have. Some of this stuff is unintuitive without a significant theoretical background.

share|improve this answer
All of my background in sampling and digital signal processing has been with regard to digital audio. I understand how a low pass filter acts to limit sounds above a certain frequency from getting into the A-D conversion. If you're sampling at 44,100hz you apply a filter that begins rolling off at about 20Khz and any response by 22Khz is pretty much gone. But with digital imaging it isn't that simple, because even with AA filters some aliasing gets through. I've read elsewhere that the filters don't try to block everything above the Nyquist because that would reduce resolution too much. – Michael Clark Feb 7 at 17:30
It seems, at least to some, that the optical AA filters in digital cameras are related more to a Kell factor than the Nyquist frequency. Scroll about 1/2 way down to post #11, and then to post #18. He seems to be saying creating an analog optical filter that does the same thing as an electrical low pass filter is not what is occurring in an optical AA filter. openphotographyforums.com/forums/showthread.php?t=14988 – Michael Clark Feb 7 at 18:09
1  
I would have to agree that the problem an low-pass filter in a camera deals with is not the same as the problem a low-pass filter in audio processing deals with. I guess the best way to put it is that an audio low-pass filter works directly with an electronic signal, where as an optical low-pass filter works on the spatial frequencies of an image signal produced by a lens. The electronic signal you are used to working with is of a different nature than an image signal. – jrista Feb 9 at 4:59
1  
@Michael: See addition to my answer. – Olin Lathrop Feb 9 at 16:39
1  
@Michael: What you say about how full color pixels are interpolated from raw sensor values is correct, but has no bearing on the aliasing discussion. Ultimately the real continuous image is still being sampled at discrete points, so a anti-alising filter before sampling is require to avoid aliases. As for your comment on algebra, it makes absolutely no sense. Of course algebra applies to higher order polynomials and 2D equations, just that it gets more complex due to there being more independent variables. – Olin Lathrop Feb 11 at 22:52
show 11 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.