There are actually three possibilities here:
- "Regular" flash
- First-curtain slow-speed sync
- Second-curtain slow-speed sync
The last is also goes by a couple of other names: trailing curtain or rear curtain. Nikon, for example, calls it "rear curtain + slow sync".
Normally, the flash is so bright that shutter speed has a negligible effect on the exposure — the exposure time is the length of the flash pulse, because the amount of light let in during the rest of the time the shutter is open is insignificant.
Slow-speed sync modes combine a flash with a long enough shutter speed to also be recorded. This gives, in effect, a double exposure: a quick sharp exposure matching the flash pulse, and a long exposure matching the shutter time.
The difference between first curtain and trailing curtain sync is simply when the flash is triggered in relation to the long shutter speed: at the beginning or end, obviously. (It's interesting to read up a bit and understand exactly how a focal plane shutter works, but isn't necessary to the basic concept of using these modes.)
The disadvantage of using a slow-speed sync mode in general is that you introduce the same difficulties of any long-exposure photography, where both camera movement and subject motion are significant factors.
First-curtain sync has a distinct problem: since the motion-freezing flash pulse comes first and the long exposure after, if the subject moves, you get a ghostly image moving away from the sharp image. If you take a picture of a moving car, for example, the blurred line of the headlights will move in front of the camera. This looks weird to most people.
Rear-curtain sync avoids this, by capturing the trail first and then the frozen subject. This is also good for, for example, people moving by firelight — the "ghost" appears to track their previous actions, which better matches our perception and memory of the flow of time.
But there is a downside as well: you have to anticipate the action by the chosen shutter time. If you've picked a two-second shutter speed, the flash exposure will come two seconds after you press the shutter button. This makes it hard to capture an exact moment intentionally, and you need better luck, anticipation skill, and planning (and even that might not be enough in all situations).