I have owned both of those lenses, as well as the "nifty fifty" - which I would assume is the 50 mm lens you already have. The Canon 24-105, even being f4, is a superior lens to the 50mm in almost every way (check out the reviews here: http://www.the-digital-picture.com/Reviews/). The 70-200 is also an excellent lens, but you're kind of comparing apples and watermelons.. they're both great, but they're very different.
For a general purpose travel lens, I would have to recommend the 24-105 - it will end up being similar in size, quality, and feel to the 16-35, and have some overlap in focal length (which may be a good or a bad thing, depending on your personal preferences). It also has enough reach for decent portrait photography, while still being wide enough to use without having to carry your second lens everywhere you go. It's often referred to as a great "walk around" lens and I tend to agree. The downside is that it will sort of "replace" your nifty fifty - which is a nice lens on it's own (very discreet in comparison to the others, while still having reasonable image quality).
Also, travelling, the 70-200 is a fair bit more conspicuous, the long, white lens makes itself a bit of a target - it definitely stands out in a crowd. The 70-200 is great if you want to take photos of people from, say, across the street, or if you want to make some portraits in a more controlled setting. Where you'll run into trouble though (or where I have in the past), is trying to get a shot of something and finding out you're too close to it, or there is something (usually people) in the way because you're so far back. I often end up switching lenses several times during the day when I have the 70-200 with me. When I don't have it, my portraits are not quite as nice, but I don't seem to notice that I have the "wrong" lens on a camera.
At the end of the day, it boils down to which one you're more likely to use. The 24-105 won't leave your camera often if you have it. The 16-35 and the 70-200 will be on and off all the time, but it will provide you with more options when deciding what look or feel you want from your images.