Let's respond to a few statements in your question:
Mode 1 compensates for camera shake in both directions and should be used for still subjects
First, IS/VC only affects camera motion, regardless of which IS/VC mode you select. It has no effect on subject motion in any mode.
Second, Mode 1 compensates for camera motion in more than just vertical or horizontal camera movements. It can also compensate for diagonal camera movements.
Mode 2 compensates only for one axis, and should be used for panning
shots
It is not so much that mode 2 only compensates for one axis, it is more that it does not compensate for a single axis along which the camera is being smoothly panned. It still compensates for other, irregular movements that are not necessarily orthogonal to the panning axis. In some implementations, IS/VC will even compensate for irregularities in the rate of movement along the axis in which the camera is being panned.
So where would I use mode 3?
The advantage of Mode 3 over Mode 1 is that the IS/VC system is centered and has equal range of movement in all directions when the shutter is actuated. In Modes 1 or 2, the system can already be at or near its limits in a particular direction of movement when the exposure begins. This limits any more compensatory movement in that same direction during exposure.
For photographers who spend a lot of time using the viewfinder to observe their subjects between exposures, like birders or other wildlife specialists, another advantage of mode 3 is that it consumes less battery power during those long periods when the viewfinder is active but no photos are being taken.
The advantage of Mode 1 over Mode 3 is that the AF system and photographer get the benefit of IS/VC when composing and focusing the shot.
... but given I'm shooting at 1/640 or higher shutter speed, does it matter at all or would I be even better off having VC turned off completely?
With APS-C cameras at 200mm, you're pretty close to the line between IS/VC being useful or not at 1/640.
Factors that will influence the decision on whether to use IS/VC and, if so, which particular mode:
- Your technique in terms of camera stability. Some folks can shoot handheld at 200mm on an APS-C camera down to about 1/160 or slower and get mostly keepers. Others might struggle at 1/640 with a FF body. The 1/(FL x CF) rule-of-thumb says 1/320 is the line for 200mm on a 1.6X Canon body.
- Your degree of pickiness. If you are displaying the results of full images fit onto a 9-10 inch tablet or a smaller phone screen the same amount of blur won't even be noticeable that would be obvious on a 24" monitor. If you plan on pixel-peeping at 100% on the large monitor and don't want to see any motion blur caused by camera movement, you probably need to use a tripod and a remote shutter release no matter what your shutter speed.
- If you are panning, Mode 2 or 'Off' should be selected. Modes 1 and 3 will "fight" the panning movement of the camera. Note that many lenses with no user selectable IS/VC/OS modes will autoswitch into "panning" mode when the IS/VC/OS sensor detects that the camera is being panned at a fairly regular rate of movement in a single direction.
These and other factors are covered in greater detail in this answer to At what shutter speed threshold does a tripod start to matter?
Panning is a skill that requires practice. The best way for you to answer your question is for you, taking into account your intended usage, to experiment with all of your options and critically observe the results to find which works most effectively.
Related: Image stabilization with monopod: on or off?
Why does Image Stabilization have a Limit?
At what shutter speed threshold does a tripod start to matter?
At what focal range does it make sense to have Image Stabilization turned on when using a Tripod?
Why does using IS with my 70-200/2.8 lens remove detail?
Does IBIS reduce image resolution? How does it compare to lens based IS?
