What should I be aware of when buying a prime lens (other than that it will fit my camera)? What specs should I pay close attention to?
The first question to ask yourself is Why do I need this prime lens? Is it because they are usually faster than zoom lenses? Is it because they are usually lighter and smaller than zoom lenses? Is it because I like changing lenses frequently (cause with prime you just have one focal length)? Is it because they are usually cheaper than zoom lenses? Or is it because they are usually sharper than zoom lenses?
The second question is What will I do with this prime lens? Do I need it because I find myself in many low-light situations? Do I need it for macro photography? Or do I need it for portraiture photography, wedding photography or another category in photography? Is it because I don't want to be lazy and to practice sticking with one focal length?
After answering these two questions you will know what exactly you need/want. Then you'll have to consider the following when you are looking for your prime lens:
The points that I've listed above can be used on other lenses too not just prime lenses, however since prime lens is a lens then it shares all the considerations for any other lens.
In conclusion, know why do you want this lens is the sole of your selection. Also when you choose a lens, don't just buy it, read reviews about it everywhere and try to search for photos that were taken using this lens. Also you can always save yourself if you can rent this lens first before you buy it.
Aside from focal length, the most important thing to consider is the maximum aperture of the lens. That's what people are often looking for when they buy primes. The max aperture is usually part of the lens description such as: 50mm f/1.8 where 50mm is the focal length and f/1.8 is the maximum aperture. The lower that "f/" number is, the better low-light performance the lens will have and the more shallow the depth of field you can get out of the lens.
Other considerations are the same as any other lens: autofocus quality, whether it has IS or can do macro (if you want/need it), if it has a rotating front element (if you plan to use a circular polarizer with it), etc.
Besides those things, there's not much else to look into regarding specs. You can get sharpness test results along with data on distortion, CA, etc on http://photozone.de and search for other reviews on Google. Flickr and it's groups can also help, people often have test data or sample photos you can view large to get an idea of the image quality you can expect -just make sure you get a large sampling of data since poor test conditions or methods can make a lens look worse than it really is.