I have a Canon camera (90D) with an inbuilt flash; when I half press
the shutter it fires the flash a bit to do metering I presume.
Actually it might be that the AF-assist is going off to help the camera focus. If you have AF-assist set in the camera menus, that's probably what you're seeing. Also, red-eye reduction uses a preflash.
The TTL (Through-The-Lens) metering preflash only happens when you fully press the shutter button, and is typically indistinguishable from the main flash burst, unless you're using 2nd curtain sync and a very slow shutter speed.
Is there a way to get the same sort of behavior to aid with metering from
an external flash? (in this case the Yongnuo YN560 IV)
The YN-560IV cannot do TTL, so if we are talking about the TTL preburst/metering, no. You would need a flash that's capable of Canon's eTTL-II communication, which requires five pins to match the five contacts on the Canon hotshoe, and the Canon communication protocol. Something like a Godox TT685-C or one of Yongnuo's TTL speedlights for Canon. (See: What are the Yongnuo flash naming conventions?)
For a first/only speedlight, I'd generally recommend one that does TTL/HSS and has a head that swivels 360º, so you can use it on-camera for event/social shooting, as well as off-camera for studio-style lighting setups. The YN-560IV is good at the off-camera, but not so great for on-camera use. There's a reason it's so cheap. (See: What features should one look for when selecting a flash?)
I've manually set the flash to its lowest setting and I took down
exposure compensation to -5 on the camera (the minimum) and the
pictures are still too bright.
It's hard to tell why this might be. But. Try taking an image without the flash at the same settings. If it's overexposed, then the flash had nothing to do with it; you're already over-exposed in the ambient.
Flash photography is like combining two exposures together: the ambient (all the light that isn't from the flash) and the flash. And the controls for the two are different.
Ambient exposure is controlled by iso, aperture, and shutter speed.
But flash exposure is controlled by iso, aperture, power, and distance.
When looking at your image you have to ascertain if the overexposure is in the ambient (things not being lit by the flash), the flash (things lit by the flash) or both, and then adjust the appropriate control.
EC only affects ambient controls.
FEC (flash exposure compensation) only affects flash power.
Is going full manual the only way option?
It isn't, but it's usually the simplest and easiest way to figure out what's going on, and the simplest way to have exact control over your iso, aperture, and shutter speed. I typically recommend mastering M mode and ambient-only exposure (exposure triangle) before attempting flash.
To me, it's kind of like going from juggling three balls (iso, aperture, shutter speed), to juggling five balls (iso, aperture, shutter speed, power, distance) while walking a tightrope (balance flash against the ambient).
I would recommend Neil van Niekerk's Tangents website for basics about using on-camera flash, flash metering, flash exposure, and balancing flash with the ambient. And also David Hobby's Strobist site (starting with Lighting 101) if you decide to get into off-camera flash.