Summer Start

by VonSchnauzer

submit your photo


Hall of Fame
View past winners from this year

Please participate in Meta
and help us grow.

Take the 2-minute tour ×
Photography Stack Exchange is a question and answer site for professional, enthusiast and amateur photographers. It's 100% free, no registration required.

I understand the purpose of the anti-aliasing (AA) filter is to prevent moire. When digital cameras first emerged an AA filter was necessary to creat enough blur to prevent moire patterns. At that time the power of in camera processors was very limited. But why is it still necessary to place an AA filter over the sensor in modern DSLR cameras? Couldn't this be accomplished just as easily by the algorithms applied when the output from the sensor is being demosaiced? It would seem that the current processing power available in-camera would allow this now much more than even a few years ago. Canon's current Digic 5+ processor has over 100 times the processing power of the Digic III processor, which dwarfs the power of the earliest digital cameras. Especially when shooting RAW files, couldn't the AA blurring be done in the post processing stage? Is this the basic premise of the Nikon D800E, even though it uses a second filter to counteract the first?

share|improve this question
    
It is not. There are already DSLRs with have no anti-alias filter, including the Pentax K-5 IIs, Nikon D800E, plus mirrorless models like the Olympus PEN E-PM2 and all Fujis (X-E1, X-Pro1). Plus they even announced fixed lens cameras without AA filter (X20 and X100S). –  Itai Feb 7 '13 at 15:46
1  
And all of those cameras show color moire at times. –  Kendall Helmstetter Gelner Feb 8 '13 at 19:34
3  
Indeed but so will other cameras. I suspect that an anti-alias filter which avoids all moire would be too strong, so manufacturers use AA filters of lesser strength. As an example, in my K-5 IIs and K-5 II comparison, moire occurs on both camera, only much more with the K-5 IIs. –  Itai Feb 10 '13 at 1:58
1  
IIRC the new Nikon D7100 doesn't have one either. –  James Snell Feb 22 '13 at 11:11
1  
And now, the Pentax K-3 has no filter but has a mode to vibrate the sensor very, very slightly during exposure to simulate one. A lot of interesting innovation in this area. –  mattdm Dec 13 '13 at 16:23

4 Answers 4

up vote 9 down vote accepted

Aliasing is the result of repeating patterns of roughly the same frequency interfering with each other in an undesirable manner. In the case of photography, the higher frequencies of the image projected by the lens onto the sensor creates and interference pattern (moiré in this case) with the pixel grid. This interference only occurs when those frequencies are roughly the same, or when the sampling frequency of the sensor matches the wavelet frequency of the image. That is the Nyquist limit. Note...that is an analog issue...moiré occurs because of interference that occurs real-time in the real-world before the image is actually exposed.

Once the image is exposed, that interference pattern is effectively "baked in". You can use software to some degree to clean moiré patterns up in post, but it is minimally effective when compared to a physical low pass (AA) filter in front of the sensor. The loss in detail due to moiré can also be greater than that lost to an AA filter, as moiré is effectively nonsense data, where slightly blurred detail could still be useful.

An AA filter is just designed to blur those frequencies at Nyquist so they do not create any interference patterns. The reason we still need AA filters is because image sensors and lenses are still capable of resolving down to the same frequency. When sensors improve to the point where the sampling frequency of the sensor itself is consistently higher than even the best lenses at their optimal aperture, then the need for an AA filter would diminish. The lens itself would effectively handle the necessary blurring for us, and interference patterns would never emerge in the first place.

share|improve this answer
    
Here is part a comment posted to photo.stackexchange.com/questions/10755/…. Do you still believe it to be accurate? If so, how is the pattern baked in until the RAW data has been demosaiced? "Ironically, at least with RAW, the theoretical nyquist limit does not always seem to be a hard limit, which is probably due to the different wavelengths of red, green, and blue light and the distribution of RGB pixels in a sensor. – jrista♦ Apr 10 '11 at 18:50" –  Michael Clark Feb 7 '13 at 13:30
1  
I believe I was talking about resolution in general there, and not directly to aliasing in the recorded digital signal. The nyquist limit is kind of a tough thing to nail down in a bayer sensor due to the uneven pattern of RGRG and GBGB rows. The spatial resolution of green is higher than the spatial resolution of either red or blue, so the nyquist limit in red or blue light is at a lower frequency than the nyquist limit in green light. The nyquist limit in a demosaiced image is kind of tough to call exactly, so it becomes a bit of a fuzzy band, rather than a concrete mathematical limit. –  jrista Feb 9 '13 at 4:25
1  
...that pattern becomes part of the image. Even if you knew the exact wavelet characteristics of the virtual image, and could produce a fourier series of then, you would have to change the orientation of the image relative to the virtual concept of the sensor to eliminate the moire "perfectly". That is a lot of excessively intense, highly mathematical work...assuming you knew the EXACT nature of the original virtual image signal and its relation to the sensor. Once aliasing is baked into a RAW, it is pretty much done, there really isn't any undoing it without softening detail. –  jrista Feb 9 '13 at 4:30
1  
I know all about the difference in frequency between red/blue and green. As for all current optical AA filters only filtering at nyquist, it really depends on the camera. Not all AA filters are designed exactly the same, and even for the same brand, different models and different lines often have AA filters that behave differently. I know that historically the 1D and 5D lines have allowed SOME frequencies just above nyquist through, however I think its a matter of balancing with lens resolution. –  jrista Feb 10 '13 at 17:12
1  
On sensors with smaller pixels, such as the Canon 18mp APS-C, the D800, the D3200, pixels are getting really, really small. Outside of a small segment if really new lenses (like Canon's Mark II L-series generation, and then, only the ones released within the last two-three years) can resolve enough detail to significantly outresolve the sensor and cause aliasing at frequencies higher than nyquist. Filter at around nyquist, and the lens itself will blur detail beyond that. I think that is part of the reason the 5D line has had an overly strong AA filter...lenses outresolve it more easily. –  jrista Feb 10 '13 at 17:15

You can't get the same effect in software. You can get somewhere nearby, given certain assumptions. But the AA filter spreads light so that it strikes multiple different coloured pixels giving you information that is absent from the no-AA filter sensor.

The Nikon D800E doesn't do anything at all to try and replicate the AA filter. If there are high frequency patterns in the image, you get moire and that's your problem - you have to deal with it!

Aliasing is at it's worse when the frequency of detail in the image is very close to the sampling frequency. For older cameras with low resolution sensors (and hence low frequency sampling) moire was a serious problem with lots of types of image detail so AA filters were strong (nothing to do with limited processing power). Now we have much higher sampling frequencies, it takes much higher frequency image details for moire to show up.

Eventually sampling frequencies will be so high the necessary high frequency object details wont make it past lens aberrations and diffraction effects, making the AA filter redundant. This is partly the reason that some MF backs lack an AA filter, super high resolution plus fashion photographers who like to shoot at f/32 with giant Profoto power packs proving lighting.

share|improve this answer
    
It seems to me the interpolation done in the demosaicing process could be modified to accomplish the exact same thing, since averaging adjacent pixels is what is done there. The Nikon D800E has two AA filter components just like other cameras, but instead of one polarizing light horizontally and the other polarizing it vertically the second is 180 degrees from the first and takes the polarized rays from the first and combines them back into one stream. See photo.stackexchange.com/questions/22720/… –  Michael Clark Feb 7 '13 at 12:37
2  
@MichaelClark No you can't get the same effect in the demosaicing process. A single point of light hitting the D800E sensor will only produce charge at one photosite. There is no way to tell what colour that light was by looking at neighbouring pixels, the information has been lost forever. The same point of light hitting the D800 sensor (with AA filter) will hit one pixel strongly and the surrounding pixels to a lesser extent. As the neighbouring pixels have different colour filters by looking at their intensities it is possible for a demosaicing algorithm to estimate the colour of the light. –  Matt Grum Feb 7 '13 at 12:53
1  
@MichaelClark The only reason the D800E has this arrangement is to simplify the manufacturing process, it's much easier to change the orientation of one of the filters at the input stage than it would be to swap two filters for a clear glass element - ultimately the filter stack has to be the same height as it has a refractive effect and modern lens designs take this into account. Simply not putting either filter on the D800E would introduce a subtle aberration in the images. –  Matt Grum Feb 7 '13 at 13:03
    
But at the same time that single point of light is hitting one sensor site, corresponding points of light are hitting all of the adjacent sensor sites and the AA filter is causing all of them to spill light onto each other. Don't most demosaicing algorithms use interpolation to compare the luminosity levels of not only the immediate pixel wells but also other near pixels wells with the same color sensitivity? Effectively, isn't blurring adjacent pixels into each other mathematically what you are doing? –  Michael Clark Feb 7 '13 at 13:22

The physics simply doesn't work that way. Aliasing irreversibly transforms frequencies past the Nyquist limit to appear as frequencies below the limit, although those "aliases" aren't really there. No amount of processing a aliased signal can recover the original signal in the general case. The fancy mathematical explanations are rather long to get into unless you've had a class in sampling theory and digital signal processing. If you had though, you wouldn't be asking the question. Unfortunately then the best answer is simply "That't not how the physics works. Sorry, but you're going to have to trust me on this.".

To try to give some rough feel that the above might be true, consider the case of a picture of a brick wall. Without a AA filter, there will be moire patterns (which are actually the aliases) making the brick lines look wavy. You have never seen the real building, only the picture with the wavy lines.

How do you know the real bricks weren't laid down in a wavy pattern? You assume they weren't from your general knowledge of bricks and human experience of seeing brick walls. However, could someone just to make a point deliberately make brick wall so that it looked in real life (when viewed with your own eyes) like the picture? Yes they could. Therefore, is it possible to mathematically distinguish a aliased picture of a normal brick wall and a faithful picture of a deliberately wavy brick wall? No, it is not. In fact you can't really tell the difference either, except that your intution about what a picture probably represents may give you the impression that you can. Again, strictly speaking you can't tell whether the wavies are moire pattern artifacts or are real.

Software can't magically remove the wavies because it doesn't know what is real and what is not. Mathematically it can be shown that it can't know, at least by only looking at the wavy image.

A brick wall may be a obvious case where you can know that the aliased picture is wrong, but there are many more subtle cases where you really don't know, and may not even be aware that aliasing is going on.

Added in response to comments:

The difference between aliasing a audio signal and a image is only that the former is 1D and the latter 2D. The theory and any math to realize effects is still the same, just that it is applied in 2D when dealing with images. If the samples are on a regular rectangular grid, like they are in a digital camera, then some other interesting issues come up. For example, the sample frequency is sqrt(2) lower (about 1.4x lower) along the diagonal directions as apposed to the axis-aligned directions. However, sampling theory, Nyquist rate, and what aliases really are is not different in a 2D signal than in a 1D signal. The main difference seems to be that this can be harder for those not used to thinking in frequency space to wrap their mind around and project what it all means in terms of what you see in a picture.

Again, no you can't "demosaic" a signal after the fact, at least not in the general case where you don't know what the original is supposed to be. Moire patterns caused by sampling a continuous image are aliases. The same math applies to them just as it applies to high frequencies aliasing into a audio stream and sounding like background whistles. It's the same stuff, with the same theory to explain it, and the same solution to deal with it.

That solution is to eliminate the frequencies above the Nyquist limit before sampling. In audio that can be done with a simple low pass filter you could possibly make from a resistor and capacitor. In image sampling, you still need a low pass filter, in this case it's taking some of the light that would hit only a single pixel and spreading it out to neighboring pixels. Visually, this looks like a slight blurring of the image before it is sampled. High frequency content looks like fine detail or sharp edges in a picture. Conversely, sharp edges and fine detail contain high frequencies. It is exactly these high frequencies that get converted to aliases in the sampled image. Some aliases are what we call moire patterns when the original had some regular content. Some aliases give the "stair step" effect to lines or edges, especially when they are nearly vertical or horizontal. There are other visual effects caused by aliases.

Just because the independent axis in audio signals is time and the independent axes (two of them since the signal is 2D) of a image are distance doesn't invalidate the math or somehow make it different between audio signals and images. Probably because the theory and applications of aliasing and anti-aliasing were developed on 1D signals that were time-based voltages, the term "time domain" is used to contrast to "frequency domain". In a image, the non-frequency space representation is technically the "distance domain", but for simplicity in signal processing it is often referred to as the "time domain" nonetheless. Don't let that distract you from what aliasing really is. And no, it's not evidence at all that the theory doesn't apply to images, only that a misleading choice of words is sometimes used to describe things due to historical reasons. In fact, the shortcut "time domain" being applied to the not-frequency domain of images is actually because the theory is the same between images and true time-based signals. Aliasing is aliasing regardless of what the independent axis (or axes) happen to be.

Unless you are willing to delve into this at the level of a couple college courses on sampling theory and signal processing, in the end you're just going to have to trust those that have. Some of this stuff is unintuitive without a significant theoretical background.

share|improve this answer
    
All of my background in sampling and digital signal processing has been with regard to digital audio. I understand how a low pass filter acts to limit sounds above a certain frequency from getting into the A-D conversion. If you're sampling at 44,100hz you apply a filter that begins rolling off at about 20Khz and any response by 22Khz is pretty much gone. But with digital imaging it isn't that simple, because even with AA filters some aliasing gets through. I've read elsewhere that the filters don't try to block everything above the Nyquist because that would reduce resolution too much. –  Michael Clark Feb 7 '13 at 17:30
    
It seems, at least to some, that the optical AA filters in digital cameras are related more to a Kell factor than the Nyquist frequency. Scroll about 1/2 way down to post #11, and then to post #18. He seems to be saying creating an analog optical filter that does the same thing as an electrical low pass filter is not what is occurring in an optical AA filter. openphotographyforums.com/forums/showthread.php?t=14988 –  Michael Clark Feb 7 '13 at 18:09
1  
I would have to agree that the problem an low-pass filter in a camera deals with is not the same as the problem a low-pass filter in audio processing deals with. I guess the best way to put it is that an audio low-pass filter works directly with an electronic signal, where as an optical low-pass filter works on the spatial frequencies of an image signal produced by a lens. The electronic signal you are used to working with is of a different nature than an image signal. –  jrista Feb 9 '13 at 4:59
1  
@Michael: See addition to my answer. –  Olin Lathrop Feb 9 '13 at 16:39
1  
@Michael: What you say about how full color pixels are interpolated from raw sensor values is correct, but has no bearing on the aliasing discussion. Ultimately the real continuous image is still being sampled at discrete points, so a anti-alising filter before sampling is require to avoid aliases. As for your comment on algebra, it makes absolutely no sense. Of course algebra applies to higher order polynomials and 2D equations, just that it gets more complex due to there being more independent variables. –  Olin Lathrop Feb 11 '13 at 22:52

These are all good answers and good information. I have a very much simplified explanation. Let's go from 2D to 1D (same concept applies).

When a frequency hits your sensor that is higher than the "max allowed frequency", it will actually create a mirror frequency into the lower side. Once your image has been sampled you will see this lower signal but the camera or your computer doesn't know if this was an actual lower signal that was really there or if it was an alias created from a signal that was too high. This information is lost. That's the reason for the "max allowed frequency" or nyquist frequency. It says this is the highest frequency that can be sampled and above it information will get lost.

an analog to audio: let's say you have your system set up where you want a frequency range from 0hz to 1000hz. to leave a little extra room you sample at 3000hz which makes your niquist 1500hz. here is where the aa filter comes in. you don't want anything above 1500hz to enter, in reality your cut-off will begin right after 1000hz but you ensure that by the time you get to 1500hz that nothing is left.

let's assume you forget the aa filter and you allow a tone of 2500 hz to enter your sensor. it will mirror around the sample rate (3000hz) so your sensor will pick up a tone at 500 hz (3000hz - 2500hz). now that your signal is sampled you won't know if the 500hz was actually there or if it's an alias.

btw. the mirror images happen for all frequencies but are not a problem as long as you are not above the nyquist because you can easily filter them out later. example input tone is 300 hz. you will have aliases at (3000 - 300 = 2700hz [and to be correct also 3000 + 300 = 3300hz]). however since you know that you are only considering up to 1000 hz these will be easily removed. so again the problem arises when the mirror images come into the spectrum that you actually want, because you won't be able to tell the difference and that's what they mean by "baked in".

hope this helps

share|improve this answer
1  
Except that "aliasing" in the context of photography is 'spacial' based on repeating patterns in the image projected onto the sensor, not on particular frequencies of light striking a single pixel well. The optical low pass filters on most camera sensors do not filter "all" frequencies above the Nyquist limit, they only filter 'most' repeated patterns at the Nyquist limit and the related multiples. –  Michael Clark Dec 13 '13 at 16:34
    
I appreciate your comment. I also wasn't referring to the frequencies of light but to the rate of change in intensity from one pixel to the next. I was ignoring the colors. I guess I was looking at it as 3 individual black and white images. Later each gets one color and overlayed together they make up all the colors. It is still hard for me to wrap my head around frequencies in images. I guess when you have a white pixel right next to a black pixel it represents high frequencies due to the fast rate of change and a light gray pixel next to a dark gray pixel is a lower frequency. –  pgibbons Dec 13 '13 at 23:14
    
That isn't exactly how demosaicing of a Bayer masked sensor works, and is one reason why I originally asked the question. –  Michael Clark Dec 14 '13 at 4:15
    
Higher frequencies in this context are repeating patterns with less distance on the sensor between each repetition. Lower frequencies are repeating patterns with more distance between each repetition. If a sensor's pixel pitch is 6µm, then patterns that repeat every 3µm would be at the Nyquist frequency. Patterns that repeat every 4µm would be below the N.F., and patterns repeating every 2µm would be above it. –  Michael Clark Dec 14 '13 at 4:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.