If you're using a high bit depth, like 16 bits per channel (not pixel), gamma doesn't matter, since it's a reversible transformation.

But, if you're using a limited depth, like 8 bits per channel, then gamma-encoding lets you store around 11 stops of dynamic range in a nominally 8-bit file.

Consider this example from Cambridge in Colour:

By applying gamma encoding, we are able to represent the original image more accurately, with the same bit depth (5, in this example).

This is achieved by using the 32 levels in a way that more closely corresponds to the human eye. In other words, it's a form of compression. JPEGs, for example, can actually store around 11 stops of dynamic range despite using only 8 bits per channel.

And like any other form of compression, it doesn't matter if you don't care about file size (and the lower speed with which you can read or write larger files). You could, in theory, use a JPEG-like format that used linear gamma, if you were willing to allocate 11 bits to each channel rather than 8.

So, to summarize, gamma is just a form of compression: it reduces the file size needed to store a certain amount of information as the eye perceives it. Alternatively, it lets you store more subtle gradations in the same bit depth.