Since your question is "What am I supposed to tell them?", I'm going to go a little bit off the literal math — which you can find at How do I convert lens focal length (mm) to x-times optical zoom?. What you should say is:
That number represents how much the lens can change, not anything absolute. The more "times-zoom" a lens has, the more compromise on image quality it has to have. Since my camera allows me to change lenses, I can have the flexibility of a huge zoom range without compromising on the results.
Then, I'd steer the conversation this way:
In fact, some very popular lenses for SLR cameras have no zoom at all, because they can be high quality and let in a lot of light without being expensive and gigantic. On the other hand, some lenses like my 18-135mm have a relatively high zoom ratio, allowing you to choose that convenience when you want it.
At that point, they may just smile and nod and back away. But they may also learn something. :)
