While the solution below did work, I found a better one. It seems like there is something fishy about the built-in color profile after all. When I switched to sRGB IEC61966-2.1 everything looks much better.
I figured out two solutions:
One way had to do with the fact that I exported to regular JPG with 8 bits of color depth. When I changed to JPEG 2000 with 16 bits of color depth it turned out just as displayed in Capture One.
The other, and much better way, as clued in by a commenter, was to just use another color profile. Turns out that the built in profile was the culprit, and when I switched to a general one like sRGB IEC61966-2.1, everything was fine.