Every paint finish is different, so there is no solid rule of thumb. Here are some basics:
Obviously, if the scratch goes through the clear coat into the color or the primer, you're not going to remove this scratch. In this case, you should get a paint stick matched from a place like paintscratch.com. Touchup, color sand the repair and re-buff the restore a final finish.
If the scratch can't be felt with your finger nail, or can just barely be felt, it is a good candidate for removal by hand or with a spot pad. Use a scratch remover polish or a fine cut compound.
If the scratch is deep, but has not gone all the way through the clear, you have to make a judgement call. These scratches can be removed, but you may remove a lot of clear doing it. It's often best to remove 80% of the scratch, making nearly invisible, than remove too much clear from the repair area.
Once you polish the clear away, you can't put it back!