I understand the primary benefit of a prime lens is the fast speed that you can get from wider aperture sizes. But when stopped down (to get a deeper depth-of-view), does it matter which lens you use?
For instance, compare a 50/1.8 with the 18-55/3.5-5.6 kit lens when both are set to f/5.6.
It obviously depends on the lenses being compared. Looking at this comparison of the EF-S 18-135mm f/3.5-5.6 IS STM and the EF 50mm f/1.8 STM, both at f/5.6 for example, the prime is sharper, even in the center of the image.
In this comparison with the EF-S 35mm f/2.8 Macro IS STM, both at f/8.0, I see more purple fringing, less sharpness and more distortion in the image from the kit lens.
No amount of stopping down will get the weight of the kit lens anywhere near of those of the two primes.
My experience (Sigma 17-70mm (which I found at least as good as my kit lens (a 18-55mm IS) vs Canon 50mm f/1.8, on an EOS 450D) is that you can still see the difference. Crop of a test shot from my window, back in 2011:
An f/1.8 lens also allows the camera body to use its more accurate AF sensor (but I don't think it mattered much here).
It depends on your requirements, the specific lenses, the camera, and the metrics you use. The only way to know is to test your lenses.
As you note, light gathering ability pretty much depends on the max F-stop, for which primes have the advantage. But when wide open, lenses tend to be softer and have more aberrations, so you might end up stopping down a lens anyway. A prime that performs unacceptably wide open has no "advantage".
Lenses tend to be softer wide open than when stopped down slightly. Some lenses "glow" when wide open in bright light. If you are stopping down a prime to the kit lens' max aperture, the prime is expected to have the advantage because the kit is operating at one of its weakest aperture settings (the other being fully stopped down).
A prime at the same "level" as the kit would normally be expected to hold the advantage at the same focal length. However, some primes are relatively soft at all apertures. For instance, I have a 50/1.4 that is softer than a 50-230/4.5-6.7 when both are set to F8. In normal photographs, the 50/1.4 offers greater creative control over aperture, but when sharpness is of great concern (or I need greater focal lengths), I know to grab the zoom.
If your "kit" lens is some 'L' equivalent glass (such as 24-105/4L), there's less room for improvement. The prime has to be that much better.
The 50/1.8 is a sharp lens to begin with, and its sharpest setting is around F5.6, where it is sharper than the kit lenses (18-55/3.5-5.6, 18-135/3.5-5.6, 24-105/4L, 24-70/2.8L). The 24-70/2.8L is quite close though.
If your prime is old, or particularly weak, a modern kit lens would likely have the advantage. Similarly, a newer prime against an old kit lens would be expected to hold the advantage. But there have been cases where a newer revision of a lens performs worse than the older model.
See Do old manual focus prime lenses give better image quality than new kit lenses?
If you are comparing a "good" copy of one lens with a "bad" copy of another, the "good" copy has the advantage, regardless of whether it is a prime or kit lens.
Full-frame lenses are at a disadvantage on crop-sensor cameras because the full imaging circle is not being used. It's like reading an eye chart at 3m vs 10m. Full-frame lens on crop sensor is reading the chart at 10m. Any lens weaknesses (contained within the imaged portion) are magnified by the crop factor.
A prime cannot beat a kit lens in terms of focal-length versatility and convenience, unless the kit is so bad as to be utterly useless at all focal lengths. – The exception being this stack of primes I've been reading about...
The difference in sharpness between a prime and kit lens may be negligible. For instance, unless you spend all your time photographing brick walls, distortion and corner sharpness aren't important beyond a certain point.
The camera also matters. If the kit lens is able to resolve beyond the sensor's capabilities, there will be no improvement with a prime, no matter how much sharper it may be.
This is just a list of tendencies. There are lots of exceptions. Also, having an advantage doesn't mean the lens is definitely better. It's like having a handicap in golf. You still have to play (test/use the lens) to win.
As a prime lens shooter, I'd like to take a different tact to answering this question by noting a possibly surprising positive: I don't need to zoom with a prime lens. That's very subjective, for sure, but I feel like I take better photos (or, at least, photos I like better) when I don't take the easy shot by simply twisting the zoom ring to frame the photo but instead am forced to move into the best position.
Additionally, prime lenses typically have a much larger focus ring than zoom lenses. Whether it's smoother or has a longer throw will definitely vary based on the lenses being compared, but, all else being equal, you'll find prime lenses have a wider focus ring. No doubt about it, it'll be easier to reach for.
I refer only only recent prosumer full frame cameras (with sensors with more than 30 megapixels) and full frame lenses.
In general, for similarly priced prime and zoom from a reputable maker, the prime (there are toasters out there) will most times show a better behavior than zooms. This is just because a zoom is optically more complex and cost more to design and make than a prime.
If we talk about top of the line prime vs a top of the line zoom, unless the prime is a toaster (and there are a few), it will behave better than the zoom. This is specially true when you get to flexibility of a wider f-stop range that delivers you superb quality, their generally lower overall weight and equal or lower price than a top of the line zoom that includes that focal length. Note that holds true most of the time, with the exception of a few very exotic primes (makers and models) or the few that have extremely wide aperture (f-stops lower than 1,0) that can cost two arms and a leg.
The same principle holds as for regular zooms, making a exceptional zoom is more complex and costly both in design and optics than a prime, and even the best zoom have trade off both quality at certain f-stops and wider apertures for its flexibility in focal range.
All that said, depending of the type of photography, the focal flexibility of a zoom is a must. Outweighing the higher weight, the lower flexibility in the range of f-stops, and even a higher price.
By the way, many users don’t know that you have to individually calibrate the focus of each of your lenses to each of your camera bodies.
Lenses and bodies are built with tolerances. If you don’t calibrate each of them you will usually suffer for front or back focusing that becomes very apparent when shutting almost wide open.
People see these focusing errors, specially the front one, and confuses it with lack of sharpness. When it is in fact a lack of focus. It is produced when the camera think is focusing perfectly but the body and lens are actually not focusing where the camera and the user thinks it’s. And the result is a photo that is out of focus.
Today most, if not all, Pro Bodies have a way to do the necessary adjustments to calibrate each individual lens by serial number.
