There are many characteristics which make better lenses better. The basic goal of a lens is to render an ideal replica of the framed scene, but because of the limitations of the real world, that's physically difficult. Lenses inevitably introduce optical artifacts not present in the scene itself. So, an important aspect is minimization of artifacts.
Good lenses are designed to get closer to that ideal image, often by using fancy, expensive lens elements made in unusual shapes and from exotic materials.
Below are some examples of some common artifacts. In some cases, the examples are deliberate test shots (although I've stayed away from photos of test targets and brick walls). In others, though, they are examples where the photographer is happily using the "defect" to artistic advantage. In fact, because some of these artifacts are part of the visual language of photography, there's a delicate balance in getting the rendering just right in a really nice lens. Still, knowing what to look for will help you be the judge of what you like to see.
Distortion
Perspective distortion, as from a wide angle lens, is simply a matter of where you stand. But lenses may introduce optical distortion as well; most common are barrel and pincushion distortion, where lines at the edges of the frame bow out or are pinched in. You'll be hard-pressed to find a cheap zoom which doesn't exhibit a visible amount of this. The good news is that this kind of distortion is easily corrected for in post-processing, but many lenses also have other, more difficult distortion ("wavy" or "mustache" patterns, for example), which can also be corrected but require knowledge of each particular lens's foibles.
Canon EF-S 18-55mm f/3.5-5.6 IS II barrel distortion. CC BY-SA 2.0 photo by cbley_.
Be careful to keep this straight from the kind of distortion that's based simply on where you stand and has nothing to do with the lens itself — that's perspective distortion. Read more about that in this question and answer.
Axial Chromatic Aberration
Axial chromatic aberration is also known as longitudinal chromatic aberration. This happens when different wavelengths of light require slightly different focus. The effect is generally visible as purple and green fringes along high-contrast edges, particularly in out-of-focus areas. This is important even in black and white photography, as it contributes to sharpness. It's unavoidable with very simple glass optics, but more expensive designs employ tricks so that red, green, and blue light wavelengths are aligned at the focal plane. Lenses which feature low chromatic aberration often have a term like "APO" in their name.
Canon EF 50mm f/1.4 USM axial chromatic aberration. Crop from CC BY 2.0 photo by Michael "Mike" L. Baird.
Transverse Chromatic Aberration
Transverse chromatic aberration is also known as lateral chromatic aberration, and is often abbreviated as "LCA", which is confusing because longitudinal CA could be abbreviated in the same way. Whatever you call it, this happens when the magnification of different wavelengths is different. This is relatively easily corrected in RAW conversion software (or even in-camera in some models), but can cause red/green and blue/yellow color fringing if not corrected for.
Severe example caused by a cheap wide-angle converter secondary lens. Crop from CC BY 2.0 photo by John Robinson.
Spherical Aberration
Simply put, spherical aberration happens when rays which pass through the edge of a lens aren't focused in the same way as rays which pass through the center. This results in a "soft lens" (but see below for a note on this). Spherical aberration can be reduced by using more lens elements or by specially-shaped elements. (Both of which increase the cost.)
Minolta Varisoft Rokkor 85mm f2.8 Soft Focus. This lens was designed with intentional spherical aberration. CC BY 2.0 photo by ming1967.
Coma
Coma is a flaw where light from an off-center object passes through the lens at an angle, and ends up focused in a sort of teardrop shape on the sensor. You may actually see oddly-shaped highlights. Generally, this is only seen on fast wide angle lenses. Lenses which have reduced spherical aberration also have reduced coma artifacts.
Zeiss Vario-Sonnar T* 24-70mm f/2.8 ZA SSM. This is actually much better than some of the other examples in this set. CC BY 2.0 photo by Jerome Marot.
Flare
Flare is light bouncing around where it shouldn't. More expensive lenses use fancier coatings to prevent reflections from the glass itself, and cheaper lenses may even skimp on internal baffles and other features designed to reduce this. (And, easily remedied but worth a mention: cheap lenses often don't come with a lens hood, the chief and simple defense against flare.)
Because it's almost unavoidable which shooting into the sun, it's entered the basic vocabulary of photography and especially of film. In fact, these days, it's often faked in video post-production.
Flare can manifest in many different ways: as a blob of glow around the light source, as rays radiating from that source, and as color-tinged rings.
Built-in lens on the Fujifilm F200EXR. CC BY 2.0 photo by Lee J. Haywood.
Ghosting
Ghosting is a type of flare, or, depending on how you want to slice it, an artifact related to flare. It may in fact be what jumps to mind immediately when you hear "lens flare". It's colored circles or polygons, usually in a line drawn from the light source — the term likens them to floating spirits. The shape directly corresponds to the shape of the aperture (and therefore the number of aperture blades, unless shot wide open).
Panasonic 7-14mm f/4.0. We can see that this lens has a 7-bladed aperture. CC BY 2.0 photo by Michael C. Rael.
Nikon Micro-Nikkor 60mm f/2.8D. This shows both obvious ghosting and other discoloration due to flare; the photographer is unhappy but I think it adds interest. CC BY 2.0 photo by Mustafa Sayed.
Veiling Glare
This is a specific kind flare which does not show up as a particular weird color, circle, or ray of light, but rather washes over the whole image. The result is overall loss of contrast. It's particularly common with older lenses; newer designs (both expensive and cheap) tend to minimize this unless you're pointing the camera directly at the sun.
Vignetting
Vignetting is fall-off of light in the corners and edges of an image. There are a number of causes, but one of them is the angle at which light hits the aperture. More expensive designs can work to minimize this.
Holga II lens on a digital camera. CC BY 2.0 photo by Soe Lin.
Field Curvature
A curved lens naturally projects a curved field, not a flat one. That's a problem because, obviously, sensors and film are flat, which means that it's impossible to get the center and edges of a frame both in focus. This can be corrected to some degree by additional elements.
Vivitar Series 1 70-210mm f/3.5. CC BY SA 2.0 photo by Andrew Butitta.
Helios 44-2 58mm f/2. CC BY SA 2.0 photo by Andrew Butitta.
In these examples you can see the "bokeh swirl" characteristic of lenses with strong field curvature. If this is an interesting look to you, and you want an even stronger effect than the aboe, check out classic Petzval lenses.
Notes on the Above
You can "stress" a lens to see its behavior under difficult working conditions by shooting directly into bright light. Lens flare is easy to see as actual bright patterns. Veiling glare is more tricky, as it produces a loss in overall contrast (which many people actually like), and that can be easily hidden in post-processing (but at a loss of shadow detail).
Curvature and vignetting can be seen in the extreme corners of an image. In many cases, like portraits, this is barely a defect and may even be preferred.
Other effects are less obvious except for in contrived situations, and may simply show up as loss of overall sharpness (and indeed may not be visible at all at web-viewing scale or in moderately-sized prints).
Shooting stopped-down usually minimizes or masks defects, so if you're looking for trouble, use the lens wide open.
The Art of Balance
The above basically all come down to science. However, there's still some art to it. One of the areas where this is most apparent is in bokeh: the rendition of out-of-focus areas. Spherical aberration is listed as a flaw above, but it's generally considered that the most pleasing bokeh is actually not the flat type produced by a well-corrected lens, but the kind that comes with slight spherical aberration. I'm not going to go into detail on that here, but see What is considered high quality bokeh?
Lensbaby lenses are very simple and produce most of the technical flaws outlined above — but it'd be beside the point to call them "not good", because they're designed to be that way.
So, the balance of the technical issues above (combined with size, weight, and cost!) and other factors cause to have a different "drawing". That's very hard to measure, and is best decided by either looking at results or listening to the subjective opinions of photographers with a practiced eye.
A Bit About Sharpness and Contrast
I want to start this one out with a disclaimer: this is overrated (and you don't have to take just my world for it). All modern lenses are decently sharp. However, since this aspect is easily measured and put into pretty charts, it features heavily in technical lens reviews. Time has proven that reviews which feature scientific-seeming numbers and boring test images get taken more seriously than ones which feature beautiful photographs, so there's a feedback loop where this gets more and more talked about.
That said, if you're cropping very tightly or printing very large, it's still important, and it's definitely true that better lenses are generally more sharp. So, please bear with me while I talk about it for a bit. Sharpness and contrast are tightly inter-related. In more technical terms, one may talk about resolution and acutance.
Resolution is the amount of detail a lens can resolve — that is, the smallest details that can be clearly imaged. This is traditionally measured by taking pictures of a target with increasingly close lines, and then seeing where they blur together.
Acutance is contrast between edges. Unsharp mask and other post-processing sharpening filters work by increasing this. Unlike on TV crime shows, software can't really add resolution, but by increasing acutance it can increase the appearance of sharpness. This different from the overall contrast of an image, which one might change with the levels or curves tool.
Note: I had previously linked acutance with the term "micro-contrast". However, I can find reputable sources defining this either as that or as resolution. Since the point, really, is to distinguish between those two properties, micro-contrast may best be avoided.
And now, let me mention the dreaded MTF charts briefly. I know that's not what you're looking for, but they're actually not that difficult and can reveal the characteristics of a lens quickly. We have more on this under How do I interpret an MTF Chart?, but the short of it is that the thick lines give you a good idea of the lens's acutance and the thin lines an idea of resolution.
Once you understand that, you can easily compare these charts in lens reviews and specifications, and you'll generally see that the lines are higher on more expensive lenses. You can see the results in actual images as well, but the charts really are a helpful tool. (The main thing you'll take away from looking at images is the point above — sharpness is often overrated.)
Build Quality and Quality Control
Build quality is simple: better lenses use better materials, and are built more solidly. Generally, this doesn't relate to image quality, but quality control can. Lenses may have optical flaws beyond the design considerations listed above. A common one is decentering, where a lens element is shifted or tilted, causing one side of the frame to focus differently from another. In one sense this is a manufacturing defect, but in modern industrial production, pretty much everything has some degree of defect, and the reliability of a random sample is basically a factor of how much money was put into the process.
Other Features
Beyond all that, it's worth mentioning that nicer lenses have nicer features, a few of which (like curved aperture blades and faster aperture) affect the lens's rendering and many others which affect their use (image stabilization, faster focus motors, weather sealing). This is some of what you pay for in a more expensive lens — not necessarily optically better, but arguably better to use. (You can read more about some of these under Is there development in the world of lenses?, where I go into a bit more detail on these things.)
