Exposure is per unit area — see Why does illuminance stay the same for a given f-stop even when focal length changes?. That means that if you measure exposure for a given shutter-speed-and-aperture on one half of the frame, it'll be the same on the other. So, that's what matters for exposure settings.
But, full-frame sensors do have an inherent low-light advantage.
Here's a way to look at it: digital sensors are 36×24mm for full frame, or 24×16mm for APS-C. When you take a correctly-exposed image, each square mm on each sensor gets the same amount of brightness. If you want to print at, say, 12×18" (mixing imperial and metric), you need to enlarge 12.7× from the full-frame sensor — or 19.05× from the APS-C one. A square mm from the full-frame camera becomes 1.27×1.27 centimeters, or 1.61cm². A square mm from the smaller-sensor camera becomes 3.63cm²! That means that in your final print, the same amount of light is spread out over 2.25× the area.
Of course, we don't print the smaller print much more darkly. Instead, we effectively amplify the brightness as we enlarge. Stretching the same amount of light into a larger area inherently gives worse results. When there's plenty of signal — lots of light — this generally doesn't matter, but when it's dim and there's a lot of noise, it does.
This is generally the reason that cameras with full-frame sensors are regarding as having about a one-stop advantage in ISO noise over APS-C. ("One stop" is 2×, of course.) In a digital camera, ISO is amplification, and for a given print size from images at the same ISO, full-frame images are literally amplified only half as much.
The issue with size of photosites is a different, technical one not actually related to sensor size and is largely obsolete with modern technology. See Do megapixels matter with modern sensor technology? for more. Even with infinitely-good technology, you can't beat the physical reality of the above. Bigger is always more light. However, in most practical cases, you can get to the point where smaller is good enough. (Otherwise, we'd all be carrying around 8×10" large-format digital cameras....)