Since you say you "have" rivets, I'm going to assume that these are existing rivets and have no snap caps or bases. Also, I'll assume they are in a visible area, so you care what they look like. Of course let me know if that's wrong.
If you plan to keep these existing rivets, and would just like to seal them, I would suggest some kind of non-silicone based caulk.
I would probably choose a polyurethane. If you are going to go with a white colored caulk, Sika 295 UV has a good reputation for not yellowing (as many white caulks are prone to do in UV light). If you want an aluminum colored caulk, Sika has a 201US. That is not quite as adhesive as 295 (it's meant for joints with a lot of movement), but it would probably do okay on a rivet.
I would try to pump caulk into the rivet hole as much as possible vs. just sticking it on top. You won't really be able to do that, because there is trapped air, but you might get a bit in if you start from one side and let a bit of air sneak out. At any rate, the idea is to have a plug of caulk inside to help hold it on.
potentially useless and boring section
Another way might be to put a teeny tiny foam rod inside the rivet hole until it is almost flush with the surface, then caulk over that and the rivet head. I'm guessing a little bit, because I don't have specific experience caulking rivets. But basically caulk does not like to have a gaping hole on one side of it. So, for example, if you are caulking a crack between a window and window trim on a house, and the crack is 1/4" wide but 5" deep, you usually put a foam "backer rod" into the crack so it is not "bottomless" (the 5"). That way the caulk behaves better. I'm not sure if a rivet tube is large enough to make a difference this way. I'm guessing that just putting some on and trying to get a little bit into the hole would be fine.
/potentially useless and boring section
I usually like to tape around things that I'm caulking (the other approach is to let it run wild but immediately clean up with solvent). In the case of a rivet I think I would tape, since there is not that much surface area and one false move with the solvent could compromise your seal.
If it's the right size, you can use a hole punch (like for notebook paper) to cut a neat circular hole in your tape; otherwise, for a number of holes (that are not that size), you can bore through a role of tape with an appropriately sized forstner bit. Or just do the old nine-million-tiny-pieces-of-square-edged-tape to make a circle. Of course it's important to prep the surface properly too.
One place that sells the Sika 295UV and the Sika 201US is Jamestown Distributors, who go by the same name on their website.