There are two main sizes of external hard disk drive: 2.5" and 3.5".
The 2.5" models are significantly less susceptible to damage due to shock and vibration/movement. These are the smaller models that plug in (usually) with just one cable and could fit in a pocket.
The 3.5" models are often described as "desktop" drives and require separate mains power, are heavier, and may even stand up vertically on your desk. These are significantly more susceptible to damage by shock and movement, and are really only designed to sit still on top of your desk.
In order to illustrate this comparison I present you the stats for a typical 2.5" and 3.5" bare drive:
Hitachi Travelstar 1TB 2.5 inch
Maximum operating shock 400G (2ms)
Maximum non-operating shock 1000G (1ms)
Hitachi Desktar 4TB 3.5 inch
Maximum operating shock 70G (2ms)
Maximum non-operating shock 300G (1ms)
Most drives will tolerate extreme temperatures (-40 to +60 celcius) while not plugged in, and relative humidity up to 95%.
So, the 2.5 inch drives are rugged enough to be carried around in luggage, and if they are inside the luggage they can probably even survive drops.
When flying don't check them, carry them on, but that goes for any equipment both because of the risk of loss and because of handling.
The main thing, really, is to always have a backup of the same data when possible - preferably on another drive in a different location. Remember that ANY hard drive, even mounted in a totally vibration-free temperature-controlled environment, can fail at ANY time.