When I increase the exposure time (say up to 20 s), it seems like my Nikon D3100 takes longer to readout the picture (no matter the intensities in the picture!). How is the readout related to the exposure time? Is that the readout time to be longer or any other process? Basically and trivially, I notice that it takes longer to display the acquired image. And from this, I thought that was something different in the readout.
There's no fundamental link between readout and exposure time.
Long exposures do tent to be noisier however, some cameras have a long exposure noise reduction feature that captures an equally long exposure but with the shutter closed, in an attempt to capture any signal generated by heat, so that signal can be subtracted from the shutter open image.
You'd know if this was the problem as it would take a whole 20 seconds for the image to be produced!
Other than that it's possible the camera is doing some time consuming software noise reduction that doesn't involve a blackframe, or that a noisy image takes longer to write to the card as it benefits less from lossless compression of the RAW or JPEG data (the extra time ought to be marginal though).
The readout time should actually be the same. What you are experiencing is the write time, where the image processor is wiring the image file (I assume RAW) to your SD or CF card. At higher ISO settings, compressing the image detail gets more difficult, as most RAW image formats use lossless compression. Since the compression is lossless, it is less effective in the face of more random detail (higher noise in the image), and thus the file sizes tend to be larger.
If you are experiencing an extremely long delay after taking a fairly long exposure (say several seconds or longer), then you might have LENR, or Long Exposure Noise Reduction, enabled. LENR will take a dark frame after the original exposure...same duration, but without opening the shutter. That can more near double the time it takes to write than it took to actually expose in the first place, as the dark frame is subtracted from the light frame before the write actually takes place.