This has a simple mathematical/geometrical explanation: In any basic geometry book you will find that the tangent to a circle is perpendicular to the line that goes from the center of the circle to the point where the tangent touches the circumference.
Following this, the tangents that touch the extremes of a diameter for a circumference, will be exactly parallel, that is, those lines won't converge at any point.
If lines that converge in a point outside the circumference ara also tangential to the circumference, the touching points will not be aligned with the center of the circumference, if you connect both touching points with the center, you will get two lines forming an angle. If you move the convergence point closer to the circumference, the before mentioned angle will be more acute.
If you make a prolongation of the lines until they intercept the diameter of the circumference, you'll find that the intersection point does not fall in the circumference but outside it, even if by a small amount.
Also, if you measure the distance between the touching points, you will find that it is in effect smaller than the circumference's diameter.
When you see an image of a sphere, the border of it is formed by rays of light that travel tangentially from the ball towhards a convergence point (presumably inside the lens) since there is a convercgence point, you are seing the same effect described above for a 2D context, but from the convergence point.
Another way of thinking about it is: if those rays of light became solid, they would conform a cone, wich would not be able to touch the ball's diameter, the only that could touch the ball by its diameter would be a tube.
The effect that you are seeing is unavoidable, and it is not related to the focal length perse, but to the distance from camera to the object. If you get further from the objects, the angle of the so called tangential lines becomes sharper, that is, closer to parallel, thus minimizing the difference you are describing.
(Sorry I do not have access to drawing tools at the moment. This is much easier to explain graphically)