Yes, they do remove a small amount of paint when they are used.
The name kind of fools people, since swirl removers don't really remove swirls. They level the paint around the swirl/scratch by removing a very small amount of the surrounding paint.
Like rollman says, it is very little and it is unlikely that you will ever damage your vehicles paint.
Charles