I don't think VR matters at all for portraits, where you can control the lighting to make sure you have a suitably fast shutter speed (1/125 or 1/250) to avoid blur due to camera shake.
AF: This depends on how good you are at focusing manually. For portraits with shallow depth of field (wide aperture), it's critical to make sure that one or both of the eyes is in perfect focus. If you can do this manually, then by all means, go for the manual focus lens.
Before buying, you can try manually focusing on one of the lenses you already have, and then check your focus by looking at the full-size image on your computer. I often find that I've missed by a bit when focusing manually, but some people are quite good.