What should I be aware of when buying a prime lens (other than that it will fit my camera)? What specs should I pay close attention to?
The first question to ask yourself is Why do I need this prime lens? Is it because they are usually faster than zoom lenses? Is it because they are usually lighter and smaller than zoom lenses? Is it because I like changing lenses frequently (cause with prime you just have one focal length)? Is it because they are usually cheaper than zoom lenses? Or is it because they are usually sharper than zoom lenses?
The second question is What will I do with this prime lens? Do I need it because I find myself in many low-light situations? Do I need it for macro photography? Or do I need it for portraiture photography, wedding photography or another category in photography? Is it because I don't want to be lazy and to practice sticking with one focal length?
After answering these two questions you will know what exactly you need/want. Then you'll have to consider the following when you are looking for your prime lens:
Focal Length: What kind of focal length you are looking for? Prime lenses' focal length can range from less than 20mm up to 600mm. The selected focal length depends on what kind of photography you want to do using this lens. For portraiture photography maybe you want to go with some telephoto focal length like 200mm for example. For landscape photography you want something wide angle so you can pick a 28mm for example.
Lens Speed: How fast do you want the lens to be? As I mentioned above, prime lenses are usually faster than zoom lenses which means they have a wider aperture which means that you can use them in low-light situations. Common apertures are f/2, f/1.8, f/1.4, and f/1.2. If you want to use a prime lens for portraiture and throw the background out of focus then the wider aperture is better (although I personally found that f/1.8 is really good in throwing the background way out of focus, I didn't try any other wide aperture so perhaps someone can share his thoughts on other apertures).
Bokeh Shape: This is an important consideration also if you are going to use this for portraiture or any other category that you want to control the bokeh shape in your picture. How the bokeh is created is something else but the basic idea is that it depends on the shape of aperture's blades. You can actually create your own bokeh if you want.
Weight: Usually the prime lens is lighter and smaller than a zoom lens because it has less moving parts. However less weight will probably means less construction quality. For example here's the popular Canon EF 50mm f/1.8 II and the Canon EF 50mm f/1.4 USM and the Canon EF 50mm f/1.2L USM. You can see that the weight of the first one is 130g (4.6 oz), the second one is 290 g (0.64 lb) and the third one is 580 g (1.28 lb). The third one is considered to have the highest quality (It's one of Canon's Luxurious lenses), the second one weight more than the first one and it has better quality than the first one. The first one weight the less among the three lenses.
Size: The size of prime lenses vary from very short (like 63mm x 15mm for Pentax pancake) to very long (like 156.5mm x 520mm for Sigma 800mm f/5.6 EX APO DG HSM) along with other dimensions maybe even more (I tried to check what's the largest available prime lens now for DSLR and I'm getting a giant 100KG monster Canon lens so I was scared to include it here). So ask yourself, do you want your lens to be compact or are you okay with more exotic large lenses? Note that your focal length may dictate the lens dimensions. For example, it will be pretty hard (from engineering and manufacturing perspective) to manufacture a 500mm prime lens to have the same dimensions as 50mm prime lens.
Quality & Sharpness: Prime lenses are usually sharper than zooms at the same focal-length and aperture because they have less moving parts and there are less variables in their design. Look for the reviews of the lens you want to buy to see if you like its quality and sharpness or not. Because sharpness is subjective you should look for photos that were shot using this lens (photos are usually part of the lens's review). You can even rent the lens and test it yourself. But you can't just ask someone "Is this lens sharp or not" cause it maybe sharp for him but not for you. Also your digital sensor type (full frame or has crop factor; more on this below) will make a difference.
Macro: If you want to use this lens for macro photography then you should look at the magnification ratio that the lens offers. The maximum and minimum aperture that the lens offer cause usually macro lenses has shallow depth of field.
Lens Stabilization: Some people think that there aren't prime lenses with a stabilization feature. While this is not true, having a stabilizing lens will probably depend on its focal length. Long lenses tend to have stabilization feature more than short lenses. The reason is simple, usually you need to stop down the shutter speed for long lenses but in the same time you want to maintain a correct exposure and a sharp image (this where stabilization shines). In the link above there is an example explaining this. Here's a list for the prime lenses with stabilization feature. Bare in mind that if yo want a prime lens with normal focal length (for example 50mm) then you maybe don't want to invest money in stabilization feature, however if you want a long lens (for example 200 and above) then you may consider including lens stabilization as part of your needs. This feature has different names in each brand, for Canon it's Image Stabilization, for Nikon it's Vibration Reduction, for Sigma it's Optical Stabilizer, for Tamron it's Vibration Compensation, ... etc. Note that some DSLRs have image stabilization feature in the camera body itself, so check you camera manual cause you don't want to pay for something that you already have.
Internal Focus VS Rotating Front Element: If you plan to use this prime lens for landscape then most probably you want to use various filters with it (Circular Polarizer filter or Neutral Density filter or others). For some types of filters, you don't want the front element of the lens to rotate while you are using these filters (for example Circular Polarizer filter or Graduated Neutral Density filter). If so then perhaps you want to buy a lens that offers Internal Focusing. Also an internal focusing lens has faster auto focus. Also if you want this lens for macro photography to photography insects then you may favor a lens with internal focus cause you don't want to wake up the sleepy insect while your lens is moving to focus. Also you may want to have a very silent auto focus motor (more about that below).
Auto Focus Vs Manual Focus: That's your personal preference. If you want Auto Focus then consider the speed of the lens auto focus motor. For example the Canon EF 50mm f/1.2L USM is way faster and better and silent actually than the Canon EF 50mm F1.8 II. Also if you kind of person that tend to use manual focus but also you want to benefit from the auto focus technology then consider a lens that has Manual Focus Override or Full Time Manual Focus. Although these lenses are usually more expensive than lenses without this feature (except for Pentax as almost all of its current lenses have this feature which called Quick Shift manual focus).
Auto Focus Motor Type: Check for the motor type that your lens is using to move the lens during auto focus. If you don't mind a noisy motor then you will find a cheaper lens than a lens with silent motor. Consider the silent motor if you are doing macro photography for insects.
Camera Format: If you own a 35mm film camera then ignore this point. If you own a DSLR then you either have a full frame or crop factor digital sensor. Most major DSLR manufacture make lenses specifically optimized for crop factor DSLR. The list includes: Canon EF-S, Nikon DX, Pentax DA, Sony DT, Sigma DC, Tamron Di-II and Tokina DX. As far as I know only Olympus and Panasonic (of the major manufactures) don't make specific lenses for crop factor DSLR, perhaps someone can confirm this. So if you have a DSLR with crop factor frame then you can look for one of these optimized lenses for your camera brand, they are usually cheaper than lenses with same specs but for full frame. Bare in mind though that you can't use these lenses on a full frame DSLR they won't work, so if you are intending to upgrade your camera body in the future, then think about this.
Budget: What's your budget for the prime lens? Prime lenses can be cheap ($90-$150) to couple of thousands of dollars. Only you can decide what you can afford. If you have a DSLR camera, you can even buy an old lens (for 35mm film cameras from ebay or keh.com) and mount it on your DSLR, but you will lose auto focus and other things but if you don't mind then you can find really cheap lenses.
Other Considerations: If you are going to use this lens in extreme weather then consider a lens that has sealing against dust and water. They won't be cheap.
The points that I've listed above can be used on other lenses too not just prime lenses, however since prime lens is a lens then it shares all the considerations for any other lens.
In conclusion, know why do you want this lens is the sole of your selection. Also when you choose a lens, don't just buy it, read reviews about it everywhere and try to search for photos that were taken using this lens. Also you can always save yourself if you can rent this lens first before you buy it.
Aside from focal length, the most important thing to consider is the maximum aperture of the lens. That's what people are often looking for when they buy primes. The max aperture is usually part of the lens description such as: 50mm f/1.8 where 50mm is the focal length and f/1.8 is the maximum aperture. The lower that "f/" number is, the better low-light performance the lens will have and the more shallow the depth of field you can get out of the lens.
Other considerations are the same as any other lens: autofocus quality, whether it has IS or can do macro (if you want/need it), if it has a rotating front element (if you plan to use a circular polarizer with it), etc.
Besides those things, there's not much else to look into regarding specs. You can get sharpness test results along with data on distortion, CA, etc on http://photozone.de and search for other reviews on Google. Flickr and it's groups can also help, people often have test data or sample photos you can view large to get an idea of the image quality you can expect -just make sure you get a large sampling of data since poor test conditions or methods can make a lens look worse than it really is.