I would first start with some standard online comparisons of the two lenses in lab conditions. Both the-digital-picture.com and dxomark.com compares these two lenses directly:
Low and behold - in these tests the prime lens does win the competition, but only slightly. They are very similar when you compare them at 14mm and 16mm. Bryan over at the-digital-picture says:
And while the 14mm L II has better image quality than the 16-35mm II at 16mm, it is not greatly better. These two lenses perform more similarly than different.
Roger Cicala, a well respected blogger and owner of a prominent online lens rental company noted:
If you want to shoot a Canon full-frame really wide, this[Canon 14mm f/2.8 L II] is the best lens you can pick up. It’s much sharper than the 16-35 f/2.8.
But honestly, why would you believe me(some random dude on the internet with lots of fake internet points), or these other websites who tested the lens in controlled environments? Why not spend $50-100 and rent the lens that you don't have, and compare it against the one that you do have? See if you don't mind losing the focal range of the zoom. Notice how you have now lost the ability to screw on standard filters. See that .05% extra sharpness and light wallet :)
Further, this is a great read to understand why these lenses might be much more difficult to make, and thus more expensive and potentially of lower quality than other lenses you use: Why are wide-angle lenses so much more expensive?