It's kinda hard to tell, but it looks like there is a hole that got plugged up with the silicone and then it got spread around a bit. Is that right? If it was a bolt hole, you might drill it out to the next size and get rid of the silicone plug. Then sand the surrounding area to get the last of it.
Flap disk seems like a reasonable choice, or a random orbital for more cautious work and final feathering. Just don't make a giant hole. If you can get all of the silicone out of there, you can fill and feather it with Marine Tex, or equivalent, white epoxy. Or, of course, fiberglass resin and cloth, and paint
BTW, I recently got a Ryobi random orbital battery
powered sander from Home Depot and it is one of the best tools I have. Exceptionally useful. Cheap too.