I also have an 07 Rav4, in my case with AWD, V6, and tow package (3,500 lb limit). You didn't say which engine you have - the V6 has ample power whereas the I4 will mean you climb hills more slowly. As others have said, trans cooling is critical and an after market cooler will help (my Rav has a trans cooling coil inside the radiator). Air bags can be added for moderate cost if you have excessive rear end sag. Brakes
in your trailer are a legal requirement in most states.
You can never officially raise your legal tow limit, but you can make your vehicle more easily cope with the additional loads. If you ever have an accident it will definitely be in your favor to be operating legally. If that is not possible you should strongly consider changing tow vehicles.
Bill in NC, who regularly tows a utility trailer