Yes, overheating is a problem.
For Canon EOS RP, the 4K mode was limited to 24fps to not overheat.
Cameras with a mirror are heavierweight, so adding a small heatsink doesn't increase the weight as much relatively speaking (400g -> 450g is more significant than 800g -> 850g), so it's probably not a surprise that Canon EOS 5D mark IV can do 4K 30fps as opposed to only 24fps, even though EOS 5D mark IV is older than EOS RP.
The issue is not a camera related problem; all brand new cameras have the same generation of technology and the heat dissipation is therefore similar. Mirrorless cameras that are designed to be small and lightweight don't have as much options for dissipating the heat, and therefore, they are more prone to overheating.