For instance, I'm talking about helpful little tips like buying a solar filter which also then serves as a cheap lens protector. Anyone have any such useful tips?
This depends on what you are photographing, the first most cheap thing I'd recommend is a book on the subject of your photographing interest (landscape, portraiture, post processing ...).
In most beginners books the accessories, that are recommended for the subject, are listed as-well. You could pick up from there the route to go.
However, if reading is not your thing, I'd say a polarizing filter (travel, landscape ...) or a flash (portraits, events ...).
My best investment is a reflector that I bought. It is a round model that is easy to fold down to a little disc. It is white on one side and silver on the other side.
You use it to lighten up shadows in the face when shooting outside.
There are many good tutorials on how to use a reflector, including this one.
If you are taking lots of portrait photographs, that tiny little flash at the top of your SLR is the WORST thing to use to illuminate your subject.
A cheap bounce flash or even wireless / remote flash or two will make a significant difference in photo quality. Just bounce it off of a light colored surface (ceiling, nearby wall, etc). This has the effect of increasing the size of your light source, which essentially makes your light source look more natural.
So, if this rule applies, then the bigger the light source, the better the photograph (MASSIVE GENERALIZATION!), then that tiny little pop up flash is about as small as you can get.
a GREAT place to start learning about this stuff is at strobist.com. Check out the "Lighting 101" area. This site has a LOT of options for people trying to put together DIY lighting kids and has lots of resources for finding off-brand strobes (flashes) effectively with your camera.
I would highly recommend a fast prime lens (fixed focal length). If you use Nikon, you can get cheap 50mm F1.8 Nikkor, or if you have an APS-C sensor (which all cheaper DSLR models do) then I would recommend the 35mm F1.8G Nikkor lens - this will give you a rough equivalent of the 'neutral' 50mm on a full frame sensor. Both of these lens are comparatively cheap.
There are several reasons I would recommend this.
1 - The shallow depth of field possible at F1.8 can make for some great photography and teach you about effective use of DoF 2 - The fixed focal length forces you to 'zoom' with your feet. :) It's a good way to force yourself to start moving around to look for the right photo. 3 - The medium focal length forces you to get up close and personal with your subjects, and not stand miles away with your 18-200 zoom. 4 - It's cheap and light, and you can always carry it in your photo bag. I carry both the 35mm and 50mm with me ALWAYS! That's how much I love them.
The other suggestion would be a proper Flash. IMHO, this has the single most significant effect on photo quality. Saying that, a TTL enabled flash ain't cheap. I would personally recommend the Nikon SB600 (if you're a Nikon user - can't comment for the other systems). The SB600 is the cheapest of the full featured flashes from Nikon, and it supports CLS (remote triggering). I will also recommend that you have a look at strobist.com and their Lighting 101 tutorials. You can also check out Joe McNally's photography and blog for a bit of inspiration - he's got some great books such as Hot Shoe Diaries that talk about the subject.
On a bit of a tangent, nowadays most professional photography that we see is always post processed for best effect. This is to such a degree that very rarely can you get a photo straight from your camera that has that same quality and feel. I suggest getting a copy of Lightroom and/or Photoshop and learning to use it. There are some great tutorials on Lynda.com for this. (Oh, and for crying out loud, SHOOT RAW! :P)
Hope this helps.
I think cheap is subjective but as far as I'm concerned a sturdy tripod and a flash have to be two of the most important accessories for a photographer to own after the camera and lens.
Next you you can think about things like a remote release (these can be picked up cheap on eBay), light reflectors, grey cards etc.
None of these are essential but they can all help to improve your shots.