With a long lens you don´t disturb people but you are left "out of the scene", while with a short focal lens you are "inside the scene" but you may disturb people. What is a better alternative for street photography?
This is very much a matter of preference and taste. Henri Cartier-Bresson was inseparable from his 50. The same holds for Jean Gaumy. On the other hand photographers like Bruce Davidson and Joel Meyerowitz seem to have a preference for wider lenses like 35 and even 28. One thing is sure: Any focal length longer than 50 is not an option. An 85 will make great snapshots, but will leave no room for creative composition.
There are three factors that you may want to consider before choosing a focal length for street photography:
Light: Shorter lenses offer a deeper field of focus at equivalent f-stops, what makes them more usable in lower light conditions. A good share of all street photography is shot at small aperture and using hyperfocal and zone-focusing techniques, so this is critical.
Space: If you are shooting in narrow streets and constrained spaces you may not have much of a choice anyway (pick a short lens).
Composition: The shorter the lens the looser the compositions you can produce. At 28mm there are simply way too many random items present within the field of view of the lens, so a meaningful composition is more difficult to achieve. Furthermore, wide lenses exaggerate perspective and it is hard to shoot anything with them that looks straight out of the camera. On the other hand, while 50mm lenses can produce great compositions in the hands of an experienced photographer, many find 50 an unnerving focal length. That is because 50s offer a rather limited field of view and almost never allow you to fit a whole scene within a single shot. So, with a 50 you need learn to make quick choices on the spot regarding what to include within your frame and what to leave out of it. This is a great photo composition exercise as it trains your eye to identify the essence of each scene in real time. But it also means that until you get used to the exercise you will likely miss some shots, and since the field of view is tight there will be little leeway for cropping afterwards.
Now, where did your criteria go, i.e. the balance between focal length and respect of personal space?
The truth is no matter what focal length the street shooters use, they almost unanimously shoot at very close range. By close range I mean distances that would be rather uncomfortable for most of us. One of the secrets of becoming a good street shooter is overcoming our fear of causing discomfort. There are obvious cases where you do not want to shoot at a very close range, but that is not a question of comfort. It is rather a function of the type of content you want to capture. Good street photography happens, in many cases, out of most photographers' comfort zone...
What I suggest is to pick your lens based on the aesthetics you want, and then find a way to use it in the context that matters to you.
One trick is to develop the right social skills to alleviate potential discomfort or eventually handle the situations of crisis. This said, I have never experienced any crisis when shooting in the streets. In most cases bringing down the camera with a smile on your face does the job. Most people smile back at you.
Speaking of gear, make sure you use a camera setup that does not come across as too intrusive or offensive. In my experience the worst choice is a large black professional DSLR. That's paparazzi material. On the other hand, some cameras are just made to be inconspicuous. Since you are a street photography enthusiast, you have probably heard about the Leica M legend (street photographer's best friend) or maybe even about Rolleiflexes and their waist-level finder (that spares you the eye-contact with subjects). I have used both of these cameras, and while I don't find them necessary for street photography, I can attest that they fulfill their job quite well. That's probably one reason why rangefinder lookalikes (e.g. Fuji x100 series, etc.) are so popular with the new generation of street shooters.
Also, note that film cameras look very harmless in general. If you are burning film on street shots, either you really are a passionate artist, or the kind of incurable pervert that goes to such an extent to satisfy his perversions. Most people go with the former and thus perceive you as an honest and harmless artist. That makes a significant difference in the way individuals react to you.
If you are interested in street photography, Eric Kim's Street Photography Blog is a classic to start with. He has guidelines for overcoming the fear of shooting in streets, and even a comprehensive guide for purchasing equipment.
I am (amongst too many other things) a 'Street Photography' enthusiast. The following is based on 'what works for me'. Tastes vary.
For an APS-C camera I very strongly recommend a zoom with a minimum focal length of around 17 or 18 mm and a top end of as much as you can afford at the quality level desired. ie 17-55 mm is an excellent start, but 17-100+ mm will not go astray, and I find an 18-250 mm very useful. Details, reasons and examples below ...
There are a number of traditional recommendations. They have their place, but if you want best possible composition results in all situations, 18 mm to xxx is preferable to less range or higher minimum focal length + a classic solution.
A fast lens (large maximum aperture) will always be useful - but is not essential.
A small lens can be useful - but is not essential. I usually use a less than tiny 17-250mm on an APS-C camera. It has seldom been of vast disadvantage. I also sometimes use a 50 mm. f /1.8 prime - about as small a lens as there is - and I find it of no great benefit size wise.
A prime lens may be "classic" - but reduces your options. If you want to be 'arty' or to pursue some subset of possible photos, then a prime may meet your needs. If you want flexibility to deal with anything you may wish to photograph, then as wide a zoom range as possible is desirable.
Street photography usually does not emphasise utter lens performance or adherence to rules - contents and inter-relationships usually predominate over absolute crispness of focus, spot on exposure of rule-matching framing. So while having the best that Carl-Zeiss can make would be very nice, a kit lens may not be too bad a start.
Starting with your 17-55 mm kit lens, equivalent to a full frame 26 - 83 mm will probably be a good initial choice.
I mostly use an 18-250 mm f/3.5-6.3 on an APSC body = full frame 27 mm- 375 mm. The top end gets used less often than the 18 mm, and most photos would be in the 18-100 (27-150 equivalent) range. Very few people will recommend a 17 or 18 mm lens and it is often cited for producing purposeful distortion, but I find it superb for super closeup 'fly on the wall' photos and for getting a good width of scene in restricted situations and when you need to act rapidly. With experience you can 'shoot from the hip' while walking through a crowd of people, or raise the camera slightly, and stand right in amongst a group of people - as invisible or visible as you wish to be. Sure - people see a tourist with a biggish camera, but you are not obviously an "in the face" photographer. Too too close and you get perspective distortion. Back off a small way and it becomes acceptable.
EXAMPLES of why a 17 mm minimum focal length (with an APS-C sensor) is desirable:
Note that the two portrait type shots below would not be considered "street photography" by most but help make the point. See link for many more examples.
The examples in this album were all taken at 18 mm focal length using an APS-C camera - in most cases a Sony A77 or A700. In many but not all cases I have added rectangles which show what you would see if a 28 mm lens had been used from the same camera position. In the examples where I have not included a 28 mm crop rectangle you will have a good idea of the effect after having seen numerous other pictures with the rectangles included.
Here is an example set of 4 photos, all taken at 18 mm and with 28 mm crop rectangles shown. (18 mm and 28 mm on an APS-C corresponds to about 27 mm and 42 mm on a full frame 35 mm sensor.) A a prime 35 mm lens would cover less area again than the 28 mm and a 50 mm even less again.
In the alleyway shot, top right, from the position used, a 28 mm lens would have vastly reduced the framing options. With even a 17-55 mm zoom you could choose from the framing shown down to an area about half the side length (1/4 the area) of the red rectangles.
In the photo of the man on the tricycle, from the position used, the difference between 18 mm and 28 mm is vast. The 28 mm cropping is too tight for even a shot focusing exclusively on the tricycle.
Similarly, the people shots at top left and bottom right offer vastly improved flexibility with a non prime lens with a 17 mm minimum focal length.
In some cases "foot zoom" will allow you to increase image coverage but this is often not possible when spontaneity or rapid action are required. In the group portrait at top left, you'd need much longer arms :-).
See also my different but relevant answer to Any reason to buy stock lens for Sony Alpha if I already have good Minolta Maxxum lenses?
Note: This is largely an answer I gave previously copied from another question with minimal editing. It got almost no notice there so given the choice I may delete it there and make this the master version - see how it goes.
For street photography, using a long focal length can be a little crippling because you might lose some of the interesting perspectives given by shorter focal length lenses.
Another thing to note is that with a short focal length lens, you can compose people into the frame and make it look like you're not really aiming at that person. If I see someone aiming a camera at my direction, I'll simply figure that they saw something interesting near me. Most people just don't believe they're interesting enough to be the main subject of a photo.
On the other hand, if I spot someone at a distance aiming a telephoto lens at me, I know immediately that I'm their target, and I'll be really weirded out by the creepy guy awkwardly trying to snap a photo of me from a distant corner.
Lastly, as long as you don't go directly up to someone and take a portrait of them, the absolute majority will just ignore it and move along. Those that don't will most likely let you keep the picture anyway, and if they don't... well, that's okay too.
P.S.: a 35mm or 50mm prime is my personal favourite for street photography
From what I understand choice of lens largely depends on where you want to be.
If you want to be in the scene, right there with your subjects, 50mm and wider lenses will be your choice.
If you want to be out of your scene, and have it largely unaffected by your presence, medium telephoto lenses is your choice. (Personally I prefer 85/1.8.)
I have my sigma 17-55 f2.8 almost always on my APS-C camera. Depending on the what you want to photograph, a pancake lens can be an option.
I just finished an international trip doing a fair amount of this kind of work using my wife's G-16, which proved to be a good tool. It is smaller than a dslr and handles quickly, though a person with large hands might find that it has too many buttons in too little space. I used the range-finder rather than the back-plate. In full-frame equiv it looks to me as though the great bulk of what I did (50+ frames) ran from approx, 28 mm to approx 60 mm. That said an ability to use the full zoom capabilities of the little G-16 was handy.