Imagine that you have a round sensor that covers full circle that your lens can expose. Your camera should be able to shoot at any aspect ratio in any orientation. You won't have to rotate your camera for portraits; just set the aspect ratio 2:3 instead of 3:2...
Your lens collection is more expensive than your camera; the sensor in your camera is only a part of the camera's cost. Why won't then camera manufacturers produce circular sensors (that would be only marginally more expensive) and provide the above mentioned conveniences?