First of all, you will save a lot of effort by adopting a software that includes worflow management and Digital Asset Management. Aperture, Lightroom, or even Adobe Bridge to name a few.
This is a bit arbitrary but works for me as a hybrid between a keyword-based and a directory-based workflow management. I use a very specific directory structure explained below.
While I do like and use metadata, keywords and smart collections, there is a drawback to them: what happens if I want to look at and search my files on a computer that doesn't have Lightroom installed for example? How can I share my photos on my network with devices like an XBox or an old piece of electronic that only supports a directory-based structure, as opposed to a database? What if I need to quickly send my photos with my phone, or create a ZIP files for friends or clients?
In my directory structure and database I keep track of both the RAW files and sRGB JPEG exported copies. The JPEG files are stored in a subfolder level by broad category first (say Celebration, Concerts, Sports, Urban Exploration). At the second level I use a strict "What - Where - When" naming convention automatically generated by Lightroom (most DAM apps support this feature). The RAW files are stored by year, then by date with a short description.
It looks like this from both the catalog/database/library view and a basic File Explorer (this is only a subset of course):
St. Patrick's Day - Albany, NY - 09, Mar/
St. Patrick's Day - Albany, NY - 09, Mar - 01.jpg
St. Patrick's Day - Albany, NY - 09, Mar - 02.jpg
4th of July - Albany, NY - 09, Jul/
Hudson Cement Factory - Kingston, NY - 10, May/
2009-03-22 (St. Patrick's Day)/
2009-07-04 (4th of July)/
2010-05-12 (Hudson Cement Factory)/
When I import my RAW files, I let Lightroom put them automatically in a RAW/year/year-month-day folder, based on the date the photos were taken on. I then add a suffix to that directory with a quick description (say St. Patrick's Day or Hudson Cement Factory, etc). I select all my RAW files, and update their metadata by setting the Scene attribute to "what" the subject is (here St. Patrick's Day or Hudson Cement Factory, which I still have conveniently in my Copy/Paste buffer). I also set the Location attributes, i.e. the City, State and Country. The earlier you set this kind of metadata, the better.
When I'm done processing, keywording and geotagging my RAW files, I export sRGB JPEG copies (and upload them to Flickr from Lightroom later on). My export preset automatically creates files that follow the "Scene - City, State - YY, Mon - Counter" naming convention, fields that I have filled by now (the date is found in the photo itself of course). I finally use Lightroom to quickly move the files to a subdir under a broad category subfolder (Celebrations, Urban Exploration, etc).
At this point what I have is a catalog/database that I can explore by metadata (date, location, scene, keywords), as well as a reasonably clear directory structure that I can use without Lightroom. This directory structure tells me what, where and when just by looking at the file names. My XBox will organize and present my photos the same way. A simple file search will quickly retrieve my photos based on these criteria.
This whole JPEG directory doesn't have to be managed by LR, but I found it pretty convenient since I still have a lot of JPG files that don't have a RAW counterpart. Why manage some, and not the others? Granted, keyword searching will return both the RAW file and JPEG file (since the JPEG file has the same keywords), but this can be easily worked out by adding a rule that will filter our either JPG or RAW/DNG files (in smart collections especially). I also use smart collections of course, to keep track of the files, versions and virtual copies I used for clients, galleries, contests and print shops.
Anyway, this worked fine so far, but I have only 6000 photos in there.