Why do my jpg phone photos show as portrait in Windows (1536x2048), but in Mediainfo and Paint Shop they show as landscape and rotated ccw (2048x1536)? It's as though they have two sets of meta data or something.
Simple: rotation is a JPEG property, and some image editors and viewers understand it, while others are dumb and just display all images starting from the top left.
Your camera is setting this based on the camera's detected orientation, but not all of the programs you are using care.
jhead, it's a simple matter of running:
jhead -autrot *.jpg
in the directory in which your files reside. Other tools (including some GUI ones) can do the same, but this is pretty simple and illustrates the concept.