You didn't say if you have one or two tanks.
If you have one tank and your regulator looks likes this:
The regulator should be mounted horizontal.
If you have one tank and your regulator looks like this:
The regulator should be mounted vertical.
If you have two tanks then your regulator probably looks like this:
This regulator should be mounted vertical. It is a changeover regulator to switch from one tank to the other.
There should also be either cast in lettering on the back stating "in" and "out" or have an arrow cast on the back to show the direction of propane
The black threaded cap is part of the regulator and should remain as is. In other words don't mess with it!
Hope this helps