EDIT:
Further investigations to cover Matt's assertions about ISO have lead to another sample image. The image below is split into four bands, two representing exposure following Matt's methodology, and two representing exposure where ISO is kept as low as possible. The assertion was that when trying to expose a scene, you set aperture and shutter then use the highest ISO possible without blowing out highlights, and correct exposure via post processing by reducing EV, to produce an image with the lowest noise. This contradicts the normal assertion that one should use the lowest ISO and adjust aperture and/or shutter to achieve a correct exposure to maintain the lowest noise possible.
The first and third bands in the image above were shot by choosing a specific shutter speed and aperture, then upping the ISO as far as possible without clipping highlights. The second and fourth bands in the image above were shot by choosing ISO 100 and a specific aperture, and adjusting shutter speed to achieve a correct exposure (no ETTR.) Both images were corrected in post production by using Lightroom's "Auto Tone" feature, which caused a reduction in the high ISO image's exposure by a little bit, and pretty much left the ISO 100 image the same.
There is a difference of three stops ISO between these two shots, and the increased noise level of the ISO 800 image is very clear in the shades. In the midtones, there is some observed increase in noise in the ISO 800 image over the ISO 100 image. In all tonal levels, fine detail has been lost to one degree or another in the ISO 800 image vs. the ISO 100 image. This can be observed by examining the remote control in its cradle, the fronts of the palm, and in any shadow tones. In the mid tones and highlights, noise levels are not specifically high enough to cause any significant degradation in print. The noise level in midtones and shadows is, however, likely to intrude onto fine details, such as one might pick up with macro photography or any photos that push the limits of resolution of a lens or sensor.
While negative exposure compensation in post processing has indeed reduced the noise level of the high ISO image to more acceptable levels, there is no question that using the lowest ISO possible will result in lower noise than even post-process negative exposure compensation of a high ISO image.
The real question is, is it always appropriate to use the lowest ISO possible? If you can expose the scene you are trying to capture without any undesirable side effects, such as blurring, underexposure, etc. then choose the lowest real ISO you can (expanded ISO settings usually achieve a lower ISO setting through digital, rather than analog, means...so using ISO 50 Expansion for example, should be avoided.) If you can not expose the scene you are trying to capture without any undesirable side effects, which might be the case when photographing sports or wildlife, photographing concerts, or doing much of any kind of indoor photography involving any action, then increasing the ISO to the minimum acceptable level that will allow you to expose your scene properly (i.e. eliminate motion blur, expose at the right level, etc.) should be chosen.
If you have the headroom, overexposing by choosing the maximum ISO you can without clipping highlights, and applying some negative exposure compensation in post processing, can help mitigate the effects of noise of very high ISO, and will be a better choice than choosing an ISO that is too low, then applying some positive exposure compensation in post processing (which will just heighten the effects of noise.)