My question is: are these good lenses sandbagging at the wider settings, or do they have a different optic system that allows them to maintain the same aperture throughout the zoom range?
Remember that when using an f-number to represent aperture, it's expressed as a fraction of the focal length, so as you zoom, the same aperture effective diameter is represented as a different number. f/2.8 at 20mm is half the aperture effective diameter of f/2.8 at 40mm. So your constant aperture zoom is not actually "maintaining the same aperture throughout the zoom range" as such. In fact, a 18-55 zoom which maintains the same aperture effective diameter throughout the zoom range would be something like f/3.5-10.7.
So neither type of zoom lens really maintains the same aperture effective diameter. Note that the effective diameter is not necessarily the true diameter of the aperture ring, either, since part of the zooming effect is that the aperture ring itself is magnified. But the effective diameter is what is relevant.
Lens designers battle to solve a number of problems including chromatic aberration, distortion, sharpness and vignetting. With a zoom lens, this is all the more difficult because they have to solve these problems not just at a single focal length but throughout the entire zoom range. However, all lens design makes compromises simply because there are so many opposing forces. For a zoom lens, the lens designers decide what aperture they can get away with at each focal length in the zoom range, without too much softening or other issues such as vignetting.
It's desirable for a zoom lens to have a much wider aperture effective diameter at the telephoto end than at the wide end, because as the image is magnified you need more light for the same amount to fall on the sensor/film. That is, you need it to be much wider just to reach the same f-number.
Cheaper zooms often just make more compromise on speed at the tele end than more expensive ones.
Constant aperture zooms like the Canon EF 17-40mm f/4.0 L you mentioned make a different compromise; they put a lot more effort into getting a wider effective aperture at the telephoto end. As a result, though, they use more glass and create a heavier lens. Since everything is a compromise too, they don't want their effort into getting a wider aperture at the telephoto end to increase the softness or vignetting at the wide end, so that limits the wide end's maximum aperture. So you get a different balance of aperture sizes compared to the cheaper, lighter "variable" (in reality actually less variation in terms of actual aperture diameter) aperture zoom and all it really depends on is what sort of trade-offs have been made in the lens design.