It should be fine. I wouldn't worry that much about a chemical cleaner such as AIO wearing down the clearcoat as much as an abrasive cleaner wearing it down. On my single stage however, AIO takes a lot of paint with it, so I have to be careful with it. In my opinion though, you should be fine to use AIO many times. I would suggest just getting rid of the minor swirls with an abrasive polish such as Megs. SFP, then AIO, then SG, then use proper washing and drying techniques to prevent swirls. Then the need to strip off the SG and polish out the swirls will be less frequent.
