Just thought that I'd post an update. I sanded down the the bumps flush, vacuumed out the holes, taped off the surrounding areas, then used a small syringe to shoot JB weld marine weld epoxy into the holes. I ended up getting the entire pack of epoxy into the holes, wear gloves its a bit messy. Next I waited the 24 hours, then sanded the epoxy bumps down flush. Drilled pilot holes and lag bolted the bracket back on. It felt like it bit really well. I also ended up having to tighten all the bolts on the other brackets as well. Seems like it worked really well and is solid. Will be taking a week long trip at the end of the month, hopefully it holds up! Thanks again for all the advice.