It seems to me that the general idea is that, once you start getting to become a decent photographer and are really moving beyond your kit lenses, you shouldn't be looking at anything slower then f/2.8
Is that the case? When is it acceptable to buy a high quality lens that is slower then that, other then for really long telephoto (at which point buying f/2.8 becomes stupid expensive). Or should you always try and stick to f/2.8 or faster?
===EDIT===
In response to some of the answers given so far, would a better question be "for which types of photos/situations can you get away with using the f/4 version of a lens instead of the f/2.8".
For example, in my (limited) knowledge, you'd definitely want to use the f/2.8 for portraiture and sports, while an f/4 would be enough for landscape and architecture photography.