How is the image impacted here?
Two sensors with different sizes having same pixel resolution are capable of giving roughly same images (not discussing noise, dynamic range and colour reproduction now). However, this is only possible in theory.
If an objective is used to photograph a black to white transition (say the brightness ratio be 1:1000, rough example) it won't create a sharp transition with same brightness ratio. An objective will create the mentioned contrast only between point distant by several pixels, like this. Ideal camera will create this brightness transition in adjacent pixels (or separated by one pixel) but no real camera would do that. If you photograph fine details with objective which is not capable of reproducing this contrast you will get gray grid of varying paleness, like this, or just even gray area if objective is not good enough.
Here is a real example, this is a comparison between sensors with roughly same resolution but one left camera features 12x9mm sensor and right camera features 36x24mm sensor.
The right camera gives much more readable text. This means that objective used with right camera can create smaller brightness transitions in relation to image height.
It happens so that optics still are dependent on sensor size: given a larger sensor one may achieve better visible resolution than with objective made specially for smaller sensor. So, using smaller sensor results in inferior resolution because of currently manufactured optics.