If I decided to take the plunge and upgrade from LR3 to LR4, does LR4 manage the migration of my catalogs for me automatically, when opening it for first time? Are there any things that break or that you wish you'd been aware of before upgrading? Obviously the safest thing to do is to take a backup of my catalogs before upgrading, but it'd be good to know if there are any "gotchas".
There is a bug in Lightroom 4 that can cause you to lose your point tone curve settings in existing images, as detailed here. This is a serious bug, if you use point curves. There are other relatively minor issues detailed at that link as well.
The behavior of Pick/Reject flags changes in Lightroom 4. Now when you flag an image, the flag is global, instead of per-collection. This can be surprising if you depend on per-collection flags.
Otherwise, I don't think there are serious issues. As you say, make backups. Your images that you've already processed will still be PV2010 (the Lightroom 3 process version) in Lightroom 4. You have to specifically upgrade them to PV2012 if you want to try the new process, because there's usually some degree of fine-tuning required. If you don't upgrade the process, they should remain identical to they way they were in Lightroom 3.
I upgraded from Lightroom 4 beta to Lightroom 4 final last night and had to upgrade all of my existing Lightroom 3 catalogs from LR3 format to LR4. The whole process was entirely painless. I have checked some of my old images and they all look the same and everything appears to have worked fine.
I don't often use the tone curve settings directly in Lightroom, I tend to use the sliders so I haven't noticed any issues as described in coneslayer's post. I do use flags but only when I'm trying to decide which images to keep/delete and that appears to be working as it did previously.
So far so good.