I have an older Quantaray 2x teleconverter for a rebel 2000 film camera will it work with a new canon dslr 77D EOS
It might, but it probably won't.
The mount will fit, but Canon's electronic communication protocols between the camera and lens are proprietary. Canon updates versions of communication protocols periodically in ways that allow older Canon products to continue working with newer ones,¹ but will often disable or reduce the usability of third party products.
Third party lenses and converters are made to work with Canon cameras by reverse engineering. This usually works fairly well with Canon cameras that are already out when the third party lens or TC is designed. Often, though, newer Canon bodies will use aspects of the Canon protocol that weren't used in previous cameras, and the reverse engineered third party products have issues working with the newer cameras.
For instance, I have a Kenko Teleplus Pro 300 2X DGX that was purchased in around 2011. It works with the 7D (2009) and 5D Mark II (2008), but has issues with the 5D Mark III (2012) and 7D Mark II (2014). When AFMA is enabled, even with a value of '0' entered, and the TC is attached between the camera and a lens, if metering or AF is activated (via shutter half-press or AF-ON button) the camera locks up and gives an error code. The only way to reset the camera is to turn it off and remove the battery. When AFMA is disabled, the TC works - but without AFMA, it's kind of a lost cause since AFMA is most needed at long focal lengths. Better results can be gotten by using the lens without TC and with AFMA applied and then cropping.
I also have a Sigma 70-200mm f/4-5.6 DG purchased in the late 1990s. It works with my film EOS cameras, but has issues with all digital EOS bodies. If any aperture other than wide open is set, when metering/AF is activated, it throws an error and locks the camera up. The camera has to be turned off and the battery removed to clear the error code.
Sometimes third party lens makers will offer firmware updates to allow their fairly recent products to work with new camera models when firmware issues crop up. This used to require a trip to the lens maker's service center, but recently many Sigma and Tamron lenses can be updated by the end user via a USB dock attached to the lens after they have downloaded the updated firmware file.
If your TC was made in the EOS film era, the reverse engineered firmware will almost certainly not be compatible with a 77D introduced in 2017. But there's nothing stopping you from trying it. If the camera gives an error code, removing the battery should reset it.
¹ At least, they are supposed to. Canon once updated the firmware of their LC-E6 chargers around 2013. The LC-E6 is the charger for LP-E6 batteries that have been around since 2008. This resulted in certain batches of older genuine Canon LP-E6 batteries being rejected by the newer version of the charger as "non-genuine" and the new chargers refuse to recharge the older batteries.