From my understanding, higher ISO gives a more grainy photo
I'm afraid your understanding is incorrect. High ISO doesn't necessarily give a more grainy photo as there are other factors involved. In some circumstances it can be the case that lowering ISO increases noise. I did an experiment a while back to prove this:
What you're seeing is exactly what you describe, one image shot at ISO 1600 unedited, versus the same photo shot at ISO 100, brightened in post to match the first image.
As you can see the ISO 100 image is considerably nosier.
What's going on is that images contain shot noise and read noise. Shot noise occurs because photons are emitted randomly by lightsources which gives rise to variations in the light hitting each part of the sensor. Read noise occurs as the analogue signal is transferred from the sensor to the ADC.
What the ISO setting on the camera does is amplify the analogue signal before readout and digitization. When amplifying the signal the shot noise gets amplified too, thus the signal to noise ratio is the same. However the read does does not get amplified as it happens after amplification. When you shoot at low ISO and brighten the image in software, the photon noise, and the read noise both get amplified, giving a higher level of noise.
So why is ISO deemed to determine image noise?
It is true that the lowest possible level of noise is achieved at the lowest possible ISO with as much light as possible hitting the sensor.
It is also true that the greatest speed for my car is achieved in fifth gear. But this doesn't mean putting it into fifth with the engine idling will increase my speed. The biggest influence on speed in the throttle position. Gearing just prevents stalling or overrevving the engine.
In the same way the biggest influence on noise is the total amount of light falling on the sensor. But you have to set the ISO to avoid over or under exposing the image.
ISO does not have a direct relationship with noise, if you have a very dim scene with little light falling on the sensor then you will have noise regardless of the ISO setting. Likewise if you have tons of light falling on the sensor you will have very little noise regardless of the ISO setting (though you might have overexposure!)
Another problem with perceptions of noise and ISO is that in any of the automatic shooting modes P/Tv/Av increasing the ISO will cause the camera to alter the shutter speed and/or aperture which results in less light hitting the sensor which means more noise. However in manual mode for a fixed shutter speed and aperture, increasing the ISO will not result in more noise being present in the image.
So why does any of this matter? After all if you want the lowest noise your camera can offer you just set it on a tripod, pick ISO 100 and leave the shutter open as long as possible before the image becomes overexposed.
The problem is that thinking about ISO first can lead to misunderstandings. For example when using the above approach in dim light with the aperture wide open, the shutter might max out at 1/30s. People remember the ISO 100 = lowest noise part, forget about the need to get a correct exposure (or are mislead by the image on the back of the LCD, which can look well exposed when you're viewing the screen in the dark) and end up underxposing the image, giving more noise than if they'd shot at say ISO 400.
It is just as correct to say the lowest possible level of noise is achieved when as much light as possible is hitting the sensor (without overexposing the image) and the ISO is as high as possible (without overexposing the image). In most cases the highest ISO possible will be 100.
Thinking about the level of light first, then the ISO avoids pitfalls when there is some limit to how much light you can get onto the sensor during the exposure.
There are other minsunderstandings that originate from thinking about ISO as the primary factor that determines noise. One such misunderstanding relates to a camera's base (minimum native) ISO. Someone with a camera Y whose base ISO is 200 might think "since ISO 200 gets me the cleanest images, wouldn't it be great to have ISO 50 like camera X". Now it may be the case that camera Y has a sensor with fantastic quantum efficiency, and very good microlenses meaning it is very efficient at capturing light, hence images become overexposed quickly, leading to a high base ISO. Camera X might have a much older sensor with poor QE, no microlenses and low fill factor. It wastes a lot of light and thus requires longer exposures. It also produces images with the same level of noise at ISO 50 as Y does at ISO 200. Just because the number is lower doesn't mean it's better.
Finally thinking about light first helps explain "ISO less" sensors, such as the latest batch of Sony sensors found in cameras like the Pentax K5 or Nikon D800. The read noise is so low that it doesn't make any difference if you amplify the signal prior to readout, meaning that you can get comparable results at many different ISO settings, proving that ISO isn't responsible for image noise.
My preferred method of shooting is in manual mode with auto-ISO. This allows me to select the depth of field and amount of motion blur that I want/can tolerate in the image, and then have the camera minimise noise for me.