For anyone else wanting to do this I figured it out.
First I got the color curve I wanted, and clicked the plus to "Save the current settings as named preset"
Then next to the plus I pressed Manage presets -> export current settings to file
Then I downloaded CurveBatch from here http://gimpfr.org/contrib_photolabo.php
Then in GIMP I clicked edit->Preferences->Folders->Plug-ins to find where plug-ins are saved.
I then saved CurveBatch here after extracting it.
Then for GIMP 2.10 I had to change the CurveBatch plug-in (photolab_curvebatch) slightly
I changed
if lines[0] == "# GIMP curves tool settings\n":
to
if lines[0] == "# GIMP 'Curves' settings\n":
As GIMP 2.10 changed the first line that indicates it is a curve file (Why anyone would think for a second to do this and break backwards compatibility for no reason I do not know)
Then I pressed Filters -> Script-fu -> refresh scripts
Then I closed GIMP and reopened it.
Then the script now showed up in filters, I pressed Filters->PhotoLab->Batch corrections->curve
I chose the folder my images are in, their extension and the curve file that I exported previously.
This then applied the curves to all of my pictures as I wanted.