The loss of contrast is due to the central obstruction; that is, the "hole of the donut" that blocks light from getting through the center of the lens.
In a diffraction-limited telescope or lens, the point spread function—basically an image of a point source, like a star—is the Fourier transform of the aperture. For a circular aperture, like most lenses, this is an Airy disk:
When you obstruct the center of the aperture, turning it into a donut, the Fourier transform changes in a way that transfers power out of the central bright spot, into the first ring. The effect is to lower the contrast of the image.
This web page has examples of the point spread function for all sorts of obstructed apertures, so you can see the effect clearly.