Is it single stage or basecoat/clearcoat? Sorry, I don't know from '96 Toyotas.
Metallic *single stage* can be a weird, and fragile, paint. But metallic basecoat/clear isn't weird at all.
I'm generally opposed to wetsanding scratches as I'd rather have thick, but imperfect paint. But it's a great way to make touch-ups (like those chips) turn out nice. If it were mine, I'd touch up the worst of the chips, wetsand them so they're pretty much level with the rest of the paint, and then polish the whole car. But I wouldn't wetsand the whole car.
If you stick with 2-3,000 Unigrit sandpaper the sanding marks shouldn't be that hard to get out. I've even done it by hand. But even with mild paper like that, you gotta be careful.