How can I avoid the reflection when I take a picture of a shiny object in bright light?
You can't remove the reflections, but you can change what gets reflected. This can be achieved by moving and shading things and lights around you.
First of all, as long as you're shooting shiny things with on-camera flash, you'll always get ugly reflection right in the middle of your picture. You can remove it only by polarizing the flash and/or lens, or by moving you main light somewhere else.
The first step is usually getting a tilt/swivel flash and bounce it off the ceiling or nearby wall. Then it's matter of finding out which one of those reflections is not visible in the object or at least looks best.
If what you're getting by this point still looks ugly, you might have to get more lights and move then around, or use the last resort -- light tent which makes all of the surroundings the same invisible white.
But not all reflections are bad -- you can use them to show the nature of your object (e.g. show that it's reflective). So keeping one or two strips of light can be beneficial.
Unfortunately I don't have any good examples of reflections of light at hand, so I'm going to post examples of dark reflections.
Therefore I think you should not be avoiding the reflections, but rather use them to help your picture. Unless it's that big image of on-camera flash in the middle, or course.
update: Lighting diagram for the first shot. I think this setup can be thought of as poor man's light tent.
There are two things I would try:
Try using a circular polarizer filter. The polarizer will block light coming from specific angles, and by turning it appropriately you should be able to block the reflection.
Control your light sources. Your best scenario would be to have a solid colored backdrop (dark colors might reduce glare as well) with lighting coming in from the side would help.
There is a nice article specifically covering photos of glass that might be helpful.
You'll want soft light, best would be an all-around softbox like a macro tent.
See also this question about photographing reflecting objects.
You need either to remove the offending light or supply much more additional light to wash out the reflection.
Removing the light can be accomplished by shading it. In this particular situation it's possible a helper could just stand nearby to block the light source.
Another way that can work with specular reflections like this is to use a polarizing filter, because the reflection tends to have a preferred polarization. The reflection won't go entirely away but it can be much reduced.
Adding light is usually done with a flash or similar source(s). Use of a diffuser or bounce is almost required, as well as appropriate positioning to avoid or at least displace the reflections. You need to add enough light that you can stop down the original exposure enough to reduce the contribution from the ambient reflections. Because this can cause related problems exposing the background, this method is to be used after careful consideration of the effect you want in the photo.
Very easy to fix. Take your flash off camera (or turn flash off, put camera on tripod, and shoot with slower shutter). Clean up your background, either by cleaning the table you're photographing on or BETTER create or buy a macro tent (around $150-$250). No tent? At the very least set up a clean backdrop. IF you have an off camera flash, put flash on outside of tent and light your tent rather than your ceramic. Last, invest in a polarizer.
You won;t be able to eliminate all highlights, but then you don't want to. Highlights help give shape and dimension. You will however be able to kill your reflections.