There are two aspects here: quality images and a budget. The budget itself has two aspects - the up front cost and the ongoing cost.
Lets look at the budget first. It is trivial to get a cheap film camera. I'll start out by saying that lenses are a wash - you're either using a camera that has interchangeable lenses (and then its the same for digital and film) or you are using a camera with an integrated lens.
Go to a camera store and get a used Canon EOS rebel or Nikon N80, and you can find them cheap. Seriously, under $50 cheap for the body (KEH lists an N80 in EX+ conniption for $76 and rebel bodies in EX condition from $12 up).
The digital camera has a larger up front cost. You are going to need to get a more expensive body than you will with a film body. It also has the additional "you're going to buy some quality digital post production software."
Going from Ilford Lab Direct's mailer, its $12/36 or $9/24 for film processing. and another $4-$5/roll for 135 film itself. Those numbers add up quite quickly as an on-going expense. You might be able reduce that some if you're willing to set up a dark room in your house (it can be rather small, but then that adds significantly to your up front costs while only marginally reducing your ongoing expenses).
Lets talk about image quality. And this is going to delve deep into the realm of opinion.
Black and white film is the best way to do black and white. Its the only way to get the grain in there. The characteristic of ilford delta grain vs tri-x vs t-max... they each have a distinct quality to them... and digital is perfectly smooth.
Filters aren't things that you can slap on after the image is taken in digital. Those frequencies of light have all been quantized and averaged. No longer do you have a sodium vapor light with its at 589.3 and 589.0 nm... you've got something that has been blended in with all the rest of the image and you can't put a didymium filter on it to selectively remove that sodium yellow light. Putting a red 25A filter does different things to the sky when you are working with the wavelengths of light rather than doing it in post production - its a 575 nm long pass filter, and that information isn't something that you can find once the light has been blended together on the sensor.
<insert ranting about push and pull and the grain and zone system here... and the grit in TriX400 at 3200>
An important factor to consider in all of this is what you want to do with those images. There is a very different appearance (IMHO) between black and white printed in silver vs black and white from even a high quality inkjet... But that's delving way into the rant and opinions (and then there are bromoil transfers, platinum prints, sepia toning and all sorts of alternative photography processes).
My belief is that you really can't capture the essence of black and white film in anything other than black and white film.
Here's what I'd really suggest doing...
Get a used Canon EOS rebel film body and a 50mm f/1.8 lens and a 52mm thread 25A filter... and go out and shoot a roll of TriX 400 with it. The camera, the filter, and the film will set you back under $100. See if you like it and if it does what you want it to. If it does, great - keep shooting and learning. If it doesn't, get a current entry level Canon EOS DSLR. New, in a kit, this is about $550.
An important bit to remember is that whatever you do, to fully use the medium isn't something that can be captured and understood with one roll of film or memory card. Truly understanding how to use your chosen media takes years of trial and error.