Two words... Pressure Sensitivity. A digitizer tablet allows you to control parameters of brushes based on how hard you press down. This is a huge advantage that a mouse can't replicate in terms of control over your work. Note that this also takes some configuration to figure out how best to maximize and may involve having to refine your touch up technique to make use of it. For very basic touch up technique, it might not make a big difference, but being able to adjust the size and amount of feathering from pressure is very key to making quick, accurate adjustments.
The added precision and absolute addressing is also worth it in the long run as it is far easier to make smooth curves with a digitizer, but that takes some practice to get used to. It also makes moving around the image faster as you can move very precisely to a given location once you are really comfortable with the tablet. (Until you are used to it, you are still basically moving the stylus until you get where you want instead of being able to go straight there.)
In general, you shouldn't expect things to be faster right away with a new input device. If you haven't used a digitizer before, it takes quite a bit of getting used to. You need to develop the ability to relate the movement of your hand on the tablet to the movement on screen. Once you get used to them, it is far faster, more accurate and more powerful than using a mouse, which is why most people swear by them.
As a simple test, if you really want to see the difference, try writing your signature with a mouse versus using the digitizer. It's something where you can make the movements without having to rely on relative position on your screen, so it will give you an idea of what it gets to be like with experience.