I bought Nikon D90 yesterday. How do I know which custom settings apply to what mode, i.e Auto, M, P etc?
The program modes (P, A, S and M) affect the shutter speed, aperture and ISO selection only as far as I know. Choosing one of these will not auto select a focus or metering mode. The camera will choose what it thinks is a sensible combination of aperture, ISO and shutter speed, but you can override this using the rear thumb dial.
The scene modes (sports, portrait, close up, landscape, etc), on the other hand, do affect the metering and focus modes, and in fact lock these down so that you can't change them. If you are in manual mode and have chosen spot metering, then switch into any scene mode, the camera will switch to matrix metering. Sports mode will select AF-A autofocus mode (why not AF-C? who knows). I think the scene modes also force white balance to Auto! Scene modes will also set Picture Controls and prevent you from using exposure compensation.
I would recommend you read up about the autofocus (AF-C, AF-S, AF-A) and metering modes (Matrix, spot, center-weighted), white balance and selecting focus points: how to set them and when to use them (a bit too much to go into here). Then use the program modes, like S (shutter) when shooting sports or moving objects, A (aperture) for most other things, and don't use the scene modes. Then you'll be able to use exposure compensation, set your white balance and so forth and not worry if the camera is doing something you don't know about.
If you want detailed information on all these combinations of settings, Thom Hogan has a good guide. The D90 manual also covers some of this.