That depends on camera make and model.
On my Canon 7D and 500D, it requires me to actually take a picture of neutral gray. Then I need to go into the menu, select custom WB, and select the picture that I took. From now on, when the camera's white balance setting is set to custom, the white balance from this picture will be used.
AFAIK, this process is common on DSLRs.
On my Canon Ixus 700 compact, I don't have to take a picture first. When I select custom WB, I have to point the camera at neutral gray, and then press the menu button. This white balance will be used until I change it into something else.