What is Aliasing and Anti-Aliasing? Is it a bad thing to have in an image? If so how to avoid it?
If you can describe your answer using examples then that would be great.
Aliasing means quite a few different things. They're all generally related (effects from digital sampling) but in different situations, it takes considerably different forms. For example, even though they're both digital graphics, what you see with a digital camera is considerably different than what you see in computer generated graphics (and both are entirely different from aliasing in digital audio).
The kind of aliasing you deal with in digital photography happens when detail in the image, when projected onto the sensor is roughly the same size or smaller than a single pixel in the sensor.
This can lead to Moire patterns and false color in the image. Just for example, let's assume there was an extremely fine white line across the image. If its width (as projected on the sensor) is smaller than the width of a sensor well, then it will show up as the color(s) of the sensor wells it happens to be projected on, rather than the actual color of the light. Even though it contains (we'll assume) exactly equal amounts of red, green, and blue, its small size prevents it from affecting all three colors of sensors equally, so it only shows up as the colors of the sensor wells it happens to strike.
Moire patterns (another artifact of aliasing) happen from interference between detail in the subject matter, and the grid of the sensor. This one is pretty easy to show:
The diagonal "stripe" patterns are from interference between the grid of the LCD I was taking a picture of, and the grid of the sensor wells in the camera. When I look at the screen, I don't see anything like that, but when taking a picture of the screen with a digital camera, it's hard to avoid.
Though most people don't take pictures of LCD screens a lot, other subjects with fine detail can as well, especially if it follows a grid-like pattern. Cloth, for one example, is notorious for causing problems, especially with cameras that don't have antialiasing filters.
As far as what you do it about it, you use an antialiasing filter. An antialiasing filter is basically just a low-pass filter, that limits the frequency you sample to something below the "Nyquist limit" for the sampling you're doing. In the case of a digital camera, that's a few sheets of glass almost directly in front of the sensor, that blur the image ever so slightly. The intent is to ensure that the finest dot of light projected on the sensor will cover at one complete red/green/blue sensor group.
As you can probably guess, this is a bit of a balancing game. If you blur the image too much, customers will be unhappy because the images aren't "sharp". If you blur it too little, they'll be unhappy because of aliasing artifacts. In most cases, you can get aliasing artifacts in extreme cases (e.g., the camera I used to take the picture above has an AA filter), and of course people still wish the pictures were sharper.
In a few cases (e.g., Nikon D800 vs. D800E) they've simply let the customer pick their poison, so to speak. To an extent, you get the same choice between typical DLSRs (most have AA filters) and medium format cameras (most don't).
I should add that without an AA filter, aliasing can lead to "false detail" -- what looks like extremely fine detail in the image, but is really just a sampling artifact. This can give a result that looks impressively sharp -- but some of that detail has almost no basis in reality at all.
Aliasing is characterised by "jagged" diagonal lines.
If you try to draw a diagonal line on a pixelated image where each pixel can be only either black or white it will appear stepped or jagged, as in this very bad example:
---- ---- ----
So if you have hard edge in a digital image you will get this effect regardless of the number of colours used.
To reduce the impact of this the line is softened or blurred so that it is spread out over a wider area, with a mid tone (or three) used to blend the line with the background:
---.. ..---.. ..---
Here I'm using the dots to represent the blurring. They would be grey to the dashes black and the background's white.
This image shows the effect:
It's normally an effect of computer generated images or those created by graphic design programs, but you can see the effect in photographs if you enlarge them too much or if it's a photograph of something with very hard edges. The lossy compression algorithms used by jpeg encoding provide a "natural" anti-aliasing effect as the sampling has the same averaging effect over the pixels.
In almost every practical sensor, parts of each pixel or pixel cluster will be more sensitive to light than other parts. If an image which is focused on the sensor has a dense pattern of light and dark areas whose spacing is close to that of the pixels or clusters, the perceived brightness of that image may be affected by whether the light parts of the pattern fall on the regions of the sensor which are more sensitive, or those which are less sensitive. In some cases, depending upon the sensor design, the perceived color of the image may also be affected.
Suppose that a camera has a sensor with a 3840x2048 grid of dots which are arranged in 2x2 clusters, with each cluster containing two diagonally-opposite green-sensitive dots, as well as one red-sensitive dot and one green-sensitive dot. One points the camera at a 385cm-wide signboard which is covered with alternating 1mm vertical white stripes and 1mm black stripes, such that the width of the signboard is precisely focused on the sensor and precisely fills it. In some areas of the sensor, the white stripes will be focused almost entirely on red and green dots; in other areas, the white stripes will fall almost entirely on green and blue dots. The net effect will be that parts of the image will appear yellow and parts will appear cyan.
If the image had been slightly out of focus, the entire sign would have appeared as a uniform gray; in fact, that is how the image should have appeared. The perfect focusing, however, will have ruined the image. The stripes of false color would be incorporated into the image in such a fashion that they could not be removed without killing a lot of real color detail.
Anti-aliasing in a sensor is a means of adding a controlled amount of "blur" to ensure that even a perfectly-focused image won't be "too perfect". If an image is already imperfectly focused, anti-aliasing will make it slightly worse, but generally not by too much. For various reasons, it's mathematically not possible to completely eliminate aliasing, but adding a tiny amount of blur can reduce it by an order of magnitude; adding a little more blur can reduce it by another order of magnitude, etc. Because amount of blur necessary to 99.9% eliminate aliasing would start to be objectionable, most cameras try to instead reduce it to the point that it is unlikely to be noticeable but may nonetheless be observed in tests which are designed to provoke it (such as the aforementioned signboard).
If one imagines that the signboard is painted with a converging pattern of stripes which are e.g. 10mm wide at the top and 0.5mm wide at the bottom, then top of the image should appear as clearly-resolved black and white stripes, while the bottom should be a uniform gray. Moving from top to bottom, the stripes should go from being black and white to being very dark gray and very light gray, then from medium-dark gray to medium-light gray, etc. until the black and white stripes become indistinguishable. Adding more blur will raise the point in the frame where the stripes cease to be recognizable as stripes, but will also minimize the unwanted aliasing effects. The quality of a camera may be judged by examining the level at which stripes cease to be recognizable as stripes, and by observing any non-uniformity in the area below that level.