Depends on the camera, adapter, and lens, but in my experience, if you are using a good simple ring adapter (no optical element) that holds the lens the correct distance away from the sensor, no. There is no image quality hit.

Canon 5DMkII + adapated Contax/Yashica Zeiss Planar T* 100/2.

Panasonic DMC-G3 (mft camera) + OM-mount Olympus Zuiko 50mm f/1.2 on a tilt adapter.
OTOH, I'm not a pixel-peeper. And Roger Cicala of lensrentals has emperical proof that adapters can cause image quality issues (although, read his article carefully. They deliberately plotted the data in a way to make the problem more self-evident (to make it really obvious if they need to service a lens), so everything looks a lot worse than it is in reality). And, as he says at the end of the article:
What Does It Mean in the Real World?
Like a lot of laboratory testing,
probably not a lot. Adapters couldn’t all stink or people wouldn’t use
them. Like a lot of tests, you can detect a very real difference in
the lab that doesn’t make much difference at all in the real world.
The only other issue to look out for when adapting to a mirrorless camera is that of sensor stack thickness. Some mirrorless cameras, micro four-thirds in particular, have a piece of glass over the sensor that can affect optical performance—particularly with wide fast lenses where the rear element is very close to the sensor. The issue was mostly discovered by Sony A7 users adapting wide-angle Leica M lenses.
See also: