I had all your same problems and concerns when I got mine. I am now going to remove the flip down wheeled jack for two reasons.
1. I have since become more skilled at backing and with just a little patience, I can usually put it right where I want and if it isn't on the dime, it's close enough.
2. removing the weight
from the tongue. I have been toting the thing around and never use it anymore.
The most it has gotten me is ONCE when the trailer was pushed into an awkward spot by two guys that happened to be there. If they were not conviniently there.. I would have backed it in.
The handle has also made a few nice dents in my bumper cover.. it sticks out so far that I lose 6 inches of turn when backing to the right.
I have a mirror for hitching.