There is a procedure to "burp" the cooling system. That's to say that you have to remove the air from the system. That air displaces coolant and can lead to overheating in that area where the coolant is blocked by the air. That may be why you can't get the full gallon into the cooling system. I don't know if the design of the cooling system on the RTV900 requires it, but my ride requires it as does many cars, trucks and off road vehicles. The basic procedure is to park the ride up hill (or jack up the front or back) so that the radiator is at the highest point. Fill the system including the reserve tank. Run the engine with the radiator cap off. Keep refilling as the air is purged until no more coolant can be added. Shouldn't take more than 30 minutes, probably much less on a small engine. Some systems are easier - just keep filling the radiator with the engine running, radiator cap off, until no more coolant can be added.
https://www.doityourself.com/stry/how-to-bleed-air-from-your-cooling-system