Many video cameras already do just what you propose. But they use a beam splitting system between the lens and the sensors which essentially cuts the amount of light reaching each sensor by roughly the same proportion as using a Bayer masked filter does.
The biggest problem with doing it with three different lenses while maintaining a very high degree of precision is the complexity and cost of creating sets of perfectly matched lenses. This is not unsolvable, but the solution is likely more expensive and computationally complex than using a Bayer mask and demosaicing.
Instead of resorting to demosaicing, what if we create a camera system that has 3 cameras, each responsible for only one of RGB? No more demosaicing required. Wouldn't it drastically improve the quality of the final image since there's no more color approximations?
Each of those three cameras would need a single-color filter in front of it in order to use the comparative differences from each to produce a "color" image. This means the same proportion of light that is absorbed/reflected by the tiny filters on a Bayer masked sensor would still be absorbed/reflected by the large single color filters in front of each of the three independent sensors. Without a different colored filter in front of each one they would all be identical monochrome cameras producing the same data.
Here's the thing about "color": there's no such thing as "color" in nature. Light only has wavelengths. Electromagnetic radiation sources on either end of the visible spectrum also have wavelengths. The only difference between visible light and other forms of electromagnetic radiation, such as radio waves, is that our eyes chemically react to certain wavelengths of electromagnetic radiation and do not react to other wavelengths. Beyond that there is nothing substantially different between "light" and "radio waves" or "X-rays". Nothing.
The cones in our retinas are made up of three different sizes that are each most responsive to a different wavelength of electromagnetic radiation. In the case of our "red" and "green" cones there is very little difference in the response to most wavelengths of light. But by comparing the difference and which has a higher response, the red or the green cones, our brains can interpolate how far and in which direction towards red or towards blue, the light source is strongest.
Color is a construct of our eye brain system that compares the relative response of the three different sized cones in our retinas and creates a perception of "color" based on the different amounts each set of cones responds to the same light. There are many colors humans perceive that can not be created by a single wavelength of light. "Magenta", for instance, is what our brains create when we are simultaneously exposed to red light on one end of the visible spectrum and blue light on the other end of the visible spectrum.