I take 'em off a few times a year, but for regular washes I just find a way to reach back in there and clean the backs/insides with them still on the vehicle. Hint: skinny, long handled wash implements. Sometimes I'll have to move the vehicle forward/back a few times to get them clean everywhere.
Now if you have BBS/monocoque style wheels, that's different. Nothing you can do except take them off unless you can reach them from the back.