I agree that a sway bar should not be needed for normal driving situations, including crosswinds, passing trucks and busses, and normal lane changes. Therefore, it is best to start without it and make sure you have a stable set-up.
From the Trailer Weights in the Real World
thread, small rear-entry models like your Compact Jr. have a tendency to be light
on the tongue. Sounds like you may have already weighed yours to verify you're carrying at least 10% of the total trailer weight on the tongue, which is a good start. Other factors- left/right loading, ball height (should be level or slightly tongue down), and how far weight is from the axle
line (weight in middle is more stable than weight on the ends, lower is better than higher)- can also affect stability. In addition, characteristics of the tow vehicle come into play, including wheelbase, center of gravity, rear overhang, etc.
One you are confident you have a stable rig under normal driving conditions, adding the sway bar is extra insurance for emergency situations.
A far more important piece of safety equipment is trailer brakes
. Do you have them?
And does your RAV4 have the tow package?