I am a computer scientist and photography is my hobby.
All the answers provided information of "traditional" compression algorithm. I should point out that there is a new algorithm that can shrink a JPEG 4x to 6x in file size without losing quality. This new compression method is specifically designed to shrink JPEG photographs.
This means that photographers now can save as much as 6 times the storage space required for their photos archives. That's why I feel this new algorithm is highly related to photography and should be brought to photographers' attention.
The technical bit, traditional JPEG compression sample across the entire photograph and apply the same "level of quality" across the photograph. The new algorithm works in the same way, but it can assign different "level of quality" to different area of the photo.
Simply put, it cuts the photo in many pieces, and saves the blurry, out of focus pieces in very low quality, and saves the sharp and colourful pieces in high quality. It is aware of the content of the photograph.
This is a bit like "variable bitrate" in audio compression.
Their website has a demo, I have used it and I stared at the two photos for minutes and can't distinguish them.
See the following techcrunch post for more details and their website:
http://techcrunch.com/2011/09/06/new-startup-jpegmini-reduces-photos-size-not-their-quality/