Aspect ratio can be anything if you're making your own cameras and coating your own plates. But if you use equipment manufactured by others, you're limited to whatever is actually made and sold. While you can crop to whatever aspect ratio you like, the full image will always be whatever is captured. Papers, frames, and other supplies are also available in standard sizes.
While there are cameras and papers that do not fit the most commonly used sizes, they hold a minority share until enough people switch. As you've noted, there's currently little market demand for manufacturers to change. So they don't.
Even if another format were to become more popular than 3:2 and 4:3, there is still pre-existing equipment that can continue to be used. There also are pre-existing images that would continue to dominate until enough new images are created.
As for why 3:2 and 4:3 are the most popular (still photo) aspect ratios...
Initially, manufacturers produced equipment and supplies that were incompatible with each other, but they eventually figured out that's a ludicrous way to compete. As time progressed, they naturally moved toward standard sizes. This has happened repeatedly with different technologies (paper, film, screens, sensors, etc).
So large format had more variety in aspect ratios. Medium format fewer. Miniature pretty much just a couple. Of course, there are always holdouts, but they hold a minority market share. (eg, square formats, panoramic formats, stereographic images, etc)
What appears to have happened (according to Wikipedia) is someone cut down a standard Kodak 70mm film stock to make the 135 format for motion pictures. Motion picture film was run vertically with a 4:3 aspect ratio (± audio track, anamorphic, etc.). Then for still cameras, someone else put two 4:3 frames together and ran the film horizontally to create the 3:2 aspect ratio. This format became popular and resisted numerous attempts at change.
By the time digital came around, 4:3 was used in consumer cameras to (probably) match the size of monitors in common use at the time. For "pro" cameras, 3:2 was (probably) used because it's what photographers expect after a century of use. There is also lots of existing equipment (paper, lenses, shutters, etc) that could be reused in the transition. (Early DSLRs were modified still cameras.)
Since there doesn't appear to be market demand for change in still photography, manufacturers have no need to change the native aspect ratio of (non-phone) cameras. However, if the demand for dedicated still photography dwindles, still photography could transition to wider native formats as more people use camcorders and phones to capture stills.