Why does it seem like large sensors are necessary for good low-light performance?
Because for the same amount of light passing through a lens a larger sensor will collect more of it.
Your tire size analogy is seriously flawed. A better analogy would be increasing the diameter of the engine's cylinders. The size of the individual molecules of the fuel/air mixture entering the cylinder stay the same in the larger cylinders, but you can put more fuel/air molecules into the cylinder and maintain the same amount of fuel/air per cubic centimeter of cylinder displacement. Thus the larger cylinder will yield more power for the same fuel/air mixture density. With a camera lens, the entrance pupil is the cylinder diameter, the f-number is the density of the fuel/air mixture. To compute the total amount of fuel/air (and thus energy) in the cylinder, the density and the displacement must both be factored in. The gear ratios and tire sizes are equivalent to the amount of amplification needed to give a higher or lower ISO sensitivity.
Surprisingly, phone manufacturers have been able to enlarge the entrance pupil enough to keep the f-number constant while increasing the sensor size.
The reason f-number is so useful is because it is a measure of field density. It is not a measure of the total amount of light passing through the lens. Rather, it is a measure of the field density of light cast by the lens on a unit area of the surface of a sensor or film at the lens' focal length. The lens projects the same amount of light per mm² of sensor area regardless of the size of the sensor. If a lens projects an image circle 44mm in diameter it is just large enough to cover a 36x24 mm FF sensor. If that same lens is placed on a camera with an APS-C sized sensor it is still projecting the same 44mm wide image circle, but the 24x16 mm sensor, with a diagonal measurement of only 29mm and a surface area only 44% the size of the larger sensor is not collecting as much of the light in the image circle as the larger sensor does.
I'm wondering why it seems to be so difficult to concentrate all of this light to the crop sensor image circle in order to get a lens that makes the same total amount of light available to the smaller sensor.
If you modify the same lens and reduce the size of the image circle cast so that all of the light collected is now projected in an image circle only 29mm in diameter (instead of 44mm), you have changed the focal length of the lens by a factor of 1/1.5X. Thus you have also changed the f-number by the same factor. You've concentrated the same total amount of light in a smaller area, thus increasing the field density. This is true whether applying it to FF vs. APS-C sized light circles or 7.5mm sensor chips vs. 5mm sensor chips. You've also required the lens to be either 50% shorter in focal length (ironically increasing the thickness of the phone that contains such a rectilinear retro-focal lens) or for the lens materials used to be significantly denser while maintaining the same amount of light transmission (increasing the cost substantially).
Why aren't there any prime lenses specifically designed for crop sensor cameras?
The f-number of a crop lens is usually at least as high as that of its full-frame counterpart and there are hardly any fast prime lenses for crop cameras at all.
There have recently been a few crop camera zoom lenses introduced that take advantage of the smaller image circle needed on such a camera to provide a lower f-number. For example the Sigma 18-35mm f/1.8 lens made for crop sensored cameras. The fastest zoom lenses made for FF cameras are generally f/2.8. The front element of a 35mm f/1.8 lens needs to be roughly the same diameter as a 50mm f/2.8 lens. Essentially what a lens such as the Sigma 18-35mm f/1.8 does is take a 28-50mm f/2.8 FF lens design and use a focal reducer to concentrate all of that light in an APS-C sized image circle. They can do so more cheaply because the smaller image circle allows them to no longer worry about correcting aberrations to the same degree that a FF 28-50mm lens with an f/1.8 sized front and casting a larger image circle would require.
There are also more affordable zoom lenses, such as the Canon EF-S 17-55mm f/2.8 or the Tamron 17-50mm f/2.8 Di II for crop sensored cameras than their 24-70mm f/2.8 FF counterparts.
The reason there are hardly any fast primes (or primes of any type for the most part) made specifically for crop body cameras is that primes made for full frame/35mm film cameras work just fine with cameras using smaller sensors. The reverse, however, is not the case. Consumer grade primes designed for FF cameras do very well on cropped cameras because the weaknesses of such lenses are almost always on the edges of the image circle that falls outside the area of the cropped sensors. And just like cameras with cropped sensors, they're a lot more affordable than premium grade FF lenses. There isn't a lot of market demand for premium grade prime lenses for cropped cameras because presumably anyone willing and able to pay for premium lenses is also willing and able, at least eventually, to pay for the benefit of a full frame camera.
What is the benefit of a larger sensor?
Assuming the same number of photosites, a larger sensor has larger photosites. This means for the same field density of light, each photosite on a larger sensor collects more total light. It also means the differences introduced by the randomness of the distribution of photons (shot noise) are averaged more evenly, thus reducing the overall impact. If the read noise per photosite is constant regardless of the size of the photosite, then a larger photosite will yield a better signal-to-noise ratio both in terms of read noise and in terms of shot noise.
And although your question attempts to exclude it, the difference in efficiency (what percentage of the photons striking the surface of the sensor actually make it to the bottom of pixel wells and are actually detected and counted by the sensor) between a larger and smaller sensor with the same number of photosites is significant. As the ratio of surface area to linear circumference increases with larger photosites, so does the efficiency. (Consider a 2x2 square has a linear circumference of 8 linear units and an area of 4 areal units. A 4x4 square doubles the circumference to 16 linear units but quadruples the area to 16 areal units.) Because light demonstrates properties of both wave energy and particle energy most of the photons that are lost, even on so called "gapless sensors", are lost at the edges of the individual photosites.