Take a look at this bike rack detail image of a Surco RV-501BR bike rack that shows how it is held to the ladder. That rack is made to fit a 1" diameter tube ladder but if you removed that clamp piece that you see against the backside of the ladder tube and replaced it with one to go around your 1.5" diameter tube you could then use it.
A little bit of an adaptive engineering adventure but not an impossible DIY project with some parts from the hardware store. What makes the adaptation possible is those clamp halves are screwed together but only the back piece has a full radius. The slight radius on the front half of that clamp can be increased in diameter with a little bit of filing or sanding time.
Keywords for the clamp fittings ..."pipe saddle clamp"