I recommend Lightroom. Wait. Doesn't Lightroom use a Library? Yes, Lightroom uses a Library, but perhaps not in the way you think.
Lightroom actually uses the directory structure you define, and will use it happily. In fact, it won't even touch your photos. It will publish to the directories as well. You can browse, tweak and yes, delete photos within your directory.
Lightroom uses a library to hold the edits and metadata for your images. The actual images are not stored in the library. Edits and metadata can be written to the image, to a new image, or to XMP sidecar files, depending on what you prefer.
Lightroom has superior image management capabilities, zero touch photo editing, and will allow you to manage the files exactly where they are, in directories of your own choosing. During the Lightroom Import process, Lightroom will even copy your originals to the directory of your choosing, because it doesn't put images in the library.
Lightroom does import, but it is not importing or even moving your images, it is simply importing the data about your image, and creating on-screen previews to aid in editing.