At least in my computers (Windows, various versions) editing a .jpg file in Photoshop and saving it creates a situation where, if there is some type of embedded thumbnail, Photoshop does not update it, later on while viewing the file in another program (Windows Image Viewer for example) the thumbnail is loaded first and shown (upscaled to fit) while the program processes the compressed jpeg data to create the full resolution image.
Most of the time, these programs are not fast enough so that the eye almost always catches the enlarged thumbnail before the actual image. In my case, I get to see the (yet) uncorrected colors, the uncropped framing, etc. (I.e. the image lacking whatever changes I made in Ps).
This is no anoyance (for me) at all, but I have seen that using "Save As" with a different name, instead of the regular "Save" avoids this situation.
What leads me to think that there is a thumbnail embedded in the metadata segment of the file is that if I transfer the edited file to a computer where no previous version of the image exists, the "original ghost" can be seen when opening the picture.
Another option that avoids the issue is saving a file with the "Save for Web" command. This uses a routine that somehow compresses the jpeg a little bit more while apparently not reducing the quality so much. Part of the algorithm includes striping most non-image data before creating the new file.
When I do this, I notice the Windows Picture Viewer takes a little extra time before showing anything at all, specially if the file was not created using "progessive" in the "Save For Web" dialog.
P.S. Windows usually creates a thumbnail database for folder that contains only pictures or mostly pictures. Editing a picture, or overwriting the file almost always makes Windows (At least XP & Vista) show the incorrect thumbnail when browsing the directory in Windows Explorer. I'm aware this is a totally different problem and seems to be unrelated to the issue described by the O.p. When this bothers me enough, I delete the hidden "thumbs.db" file in the offending folder then switch the view mode to anything but Large Thumbnails and then back to that. This forces the creation of a new thumbs.db file that should use the actual data from the files in the folder.