This is a very personal choice and hard to answer for you. To help you make the choice, I would ask how often you find the 105mm to be a limiting factor. For family portraits (and by that I assume you mean some posed portraits and a lot of snapshots?) I could see using a moderate telephoto at times, but more often something in the 50-90mm range would be more useful. On a full frame at times I've used 200mm for head shots, but on a DX body, the 300mm focal length is 400mm equivalent. Do you need that reach?
The 70-300mm VR is a good versatile lens, but if you want crisp shots, I'd probably stick with the primes.
I have the 50mm 1.4G and it's great. I think for the money, the 1.8G is probably better value. Rather than the 70-300mm VR, for almost the same price you can probably get a 50mm 1.8G and an 85mm 1.8G (or 1.8D). You get several more stops of light with the primes, so they'll be more useful indoors, sharp, nice bokeh. Loads of advantages, as long as you don't mind switching out lenses.
If you have lightroom or another library with your family photos, see if you can get an idea of what focal lengths you use most often. If you're shooting a lot at 105mm then perhaps you do want some more reach. Otherwise I'd go with primes.