What is considered high quality bokeh and what is considered low quality bokeh?
Good or bad bokeh is in the eye of the beholder. Michael Clark thinks the highlights have to be round, usually meaning a fully open aperture, but I don't agree. I don't mind 7-gon or 9-gon highlights, though round one do look nicer.
An aspect of bokeh which bothers me more than the general shape is the light distribution. For me the best bokeh is when the highlights are lit evenly, , contrary to brighter at the center or the rim. An extreme example of this is a mirror lens bokeh. Since the bokeh highlights mirror (no pun intended) the shape of the lens' aperture, and since in a mirror lens the center is blocked the highlights are donut shaped. Take a look at this example:
(I don't know who the photographer is.) Here an otherwise beautiful picture is IMO ruined by the mirror lens bokeh, in particular on the boy's bermuda shorts.
Like I said, it's a personal thing.
In general "high quality" bokeh is viewed by most photographers as out of focus areas that are smooth rather than harsh. Most lenses that produce smooth bokeh also produce out-of-focus highlights that are round rather than an equiangular regular convex polygon. Since lenses that produce round out of focus highlights with soft edges also tend to produce smoother out of focus areas in general this is one of the easiest ways to evaluate the quality of bokeh that a lens produces.
"Low quality" bokeh is viewed as out of focus areas that have harsh transitions from the blur of one out-of-focus object to another. Lenses with harsh bokeh also tend to to produce out of focus highlights that are not round. Some may also produce round out-of-focus highlights that have very hard edges or are even brighter near the edge than the center which also tends to make the bokeh very harsh for all out-of-focus areas.
Compare the shape of a five-sided regular polygon here:
to the shape of a nine-sided regular polygon here:
Even with straight sides it is obvious how much closer to circular the nine sided figure appears than the five-sided one. Lenses that produce "high quality" bokeh usually have aperture blades that are curved which allows the opening to be even closer to a circular shape than those with straight edges.
The shape of the bokeh is determined by the shape of the aperture diaphragm. Circular bokeh is possible when the diaphragm blades are curved and form a near-circle at a particular aperture setting. Most lenses that can produce round bokeh have nine or more individual aperture blades that are curved. Lenses with fewer blades tend to produce rougher bokeh. Often the aperture blades in lenses with fewer of them are not curved. Even if they are curved the angles change too much from one aperture setting to the next so there is only one setting that allows the curve of each blade to match the curve of the adjacent blades and form an almost circular opening.
There are some lenses which do not place any aperture blades in the light path when set to the widest setting. The Canon EF 50mm f/1.8 II is one such lens. The bokeh is much smoother when shot wide open at f/1.8 than when shot stopped down with five aperture blades in the light path. As with most lenses, shooting the EF 50mm f/1.8 wide open to get smoother bokeh also means sacrificing some in-focus sharpness compared to using narrower apertures.
High quality is when it fits the vision and intent of the photographer and it enhances the aesthetics of the image. Bad quality is when it subtracts from the overall quality/aesthetics of the photo and for technical reasons does not match the photographer's intent.
In general, the intent of bokeh is to make a smooth background against a sharp foreground. Good bokeh lenses make a smooth background, while bad ones make a "busy" bokeh. This takes attention away form the subject, which is generally considered bad.
Sometimes the photographer might want to use the "bad" one for a particular artistic reason. The important thing is that it is done as an informed choice. examples of busy bokeh that people seek are some cult vintage lenses that have a "swirly" bokeh, like the Russian ones. While e.g. Zeiss and Pentax vintage lenses are very smooth.
Then there's also the case of "bokeh balls", some are very uniform (good) or dissipate towards the edges (good), others are doughnuts or have dots in them (bad). Some are really round , some are pentagons, hexagons, and some are like shurikens then stopped down a bit. In general round ones are desired. Again creative choices may differ.
You can check out differences in bokeh looks on different sites:
There's examples of busy and creamy bokehs amongst these.