Looking at RAV4 and wonder if the door that is not a hatch is a real bother.
I've owned the two latest generations of RAV4 - door swings open on both. Hasn't really bothered me.
When towing, bother factor depends primarily on the height of your hitch and ball, as the square receiver on the RAV4 is naturally pretty low so you typically use a draw bar with a 4-5" raise in order to keep the trailer horizontal, and that's what can potentially interfere with the door. The door on the most recent RAV version is a bit higher and I picked my draw bar so it clears the door without any problem even with the raised latch on the coupler. Depending on what you keep on your trailer tongue, it can even open wide enough to allow access to the stuff inside even with the trailer hooked up.
What I like the most about the swinging door is that you can install a bike carrier onto the spare tire, and retain the ability to open the door without removing the bikes.