There's a bit more than a single question in your query. Focusing on the first one:
Obviously ... [the] Nikon Coolpix P900 ... does not range from actual 24 mm to 2 meters in physical length, but what kind of optical systems do manufacturers use to bring optical zooms to such high (83x) levels?
Firstly, don't confuse the 35mm equivalent focal lengths (24mm–2000mm) with the actual focal lengths of the P900's lens, which ranges 4.3mm–357mm. For more on this, see the related question, What is crop factor and how does it relate to focal length?
Still, you are right, obviously the physical length of the P900's lens is not so short as 4.3mm, nor as long as 357mm. So we need to address the fundamental question, what is focal length?
If you could replace the P900's wide-angle focal length with single symmetric lens glass (like a magnifying glass) with the same magnification as the P900's wide end, then parallel light rays coming into that single-element lens would focus at a point 4.3mm, nearly one-sixth of an inch, beyond the lens. That's what focal length means.
The real lens of the P900 isn't that short. All real-world lenses have multiple optical elements in them. These elements work together to bend and "unbend" (diverge) the light rays several times. The point of these multiple elements (and groups of elements) are:
- to allow the lens to focus, rather than moving the entire lens assembly in or out to focus (these elements comprise the focus group(s));
- to allow the lens to zoom — i.e., change focal length;
- to control aberrations such as chromatic aberration (which is a natural consequence of light bending through materials with different refractive properties).
So lens designers add a lot of elements to make the lens more useful than a narrow range of operating conditions. That adds length to the lens assembly, but in the case of wide angle lenses, it is optically "equivalent" (using that word loosely) to a simple single-element lens with a focal length shorter than the real-world lens's physical length.
Similarly, at the other end of the P900's zoom range, at 357mm focal length, the lens is physically less than 357mm long. That's because the lens has a telephoto group, a group of elements that allow a lens to be physically shorter than its thin lens –equivalent focal length would dictate. See also, What is the difference between a telephoto lens and a zoom lens?
Now, as far as, how do they specifically get zoom ranges as high as 83x (or even more, such as the 125x zoom in Nikon's recently announced P1000), well... science. Magic. A little bit of both?
These high zoom hybrid cameras are relatively cheap compared to regular DSLR super telephoto lenses, so it does not look like they use the same expensive optical elements at all. They manage to pack 83x in 20cm or so, while the best 600mm DSLR lenses are about twice longer for about 12x, so I wonder how they do it, how much light is lost in the process, etc.
Regarding comparing zoom amount vs. lens length, remember to compare like vs. like. In this case, the Square-cube law (Wikipedia) comes into play: as an object is scaled in size by a factor S, its surface area is scaled by S², and its volume is scaled by S³.
The P900 has a crop factor of 5.6, meaning the linear scale factor between cameras like the P900 with a 1/2.3" sensor and 35mm full frame cameras is S = 5.6 (from P900 to 35mm FF). So, to create "equivalent" optical systems (as far as the geometry of the optics is concerned), the scaled-up P900-type 83x lens, but made for a 35mm FF body, would:
- have roughly a 5.6 times larger diameter, and
weighs roughly 5.6³ = 175 times the weight of the P900 lens!1
Note 1: The lens probably wouldn't weigh quite that much; a simple S³ scaling implies all components, including focusing motors, lens tubes, focusing helixes and controls, etc., scale their wall thickness 5.6 times. That's not necessary, there'd be plenty of weight to shave. But, from an optics standpoint (without changing the optical formula), the weight of the glass would scale by S³.
And note that even just the length of the ~20cm lens scales up by S to about 112cm — that's over a meter long.
I don't even want to know how the cost would scale, but 175x the $1000 cost of the new P1000 actually wouldn't be totally out of line for such a ludicrously monstrous beast if it were for 35mm full frame bodies.
After that, there have been several different one-of-a-kind -type lenses that went for astronomical prices, but let's not get ridiculous (!).
So there are obvious nonlinear cost benefits to scaling down. But since the P900 and P1000 aren't aimed at the high-end professional or prosumer market, they can make further cost-savings decisions, such as:
- using lower-quality optical glass in some of the lens elements;
- reducing the number of optical coatings (such as low-dispersion and anti-reflective);
- eliminate weather sealing;
- provide less warranty coverage;
- and all of the other usual suspects when companies engage in market segmentation.