40 lbs. is pretty big difference when it comes to rotating mass. I know you weren't talking about drag racing, so that's not where I'm headed.
Actually, in regards to the drag racing, every 100lbs does equate to 1/10th of a second, but that's not rotating mass or "un-sprung" weight, that's curb/static weight which is a different deal all together.
Anyway, in some cases, when going to a performance wheel/tire package, you are increasing the weight without realizing, which is why I as asking how much your stockers weighed. You might start off with a lighter wheel, but most people increase the diameter and width of their wheel/tire, which means you are increasing the size and weight of the tire. In those cases, a lighter wheel is a must to compensate for the much heavier tire in order to bring you back to "zero".
In general, a lighter wheel/tire combo will give you better peformance, hence why race cars use the lightest combo possible. That being said, a lighter wheel/tire will make your car ride rougher for everyday use, and will also be easier to damage. There's a balance here that you have to find.
I asked what your stock wheels weigh because I think, if it were my car, I'd try to stay close to what the stock wheels are, hopefully losing a pound or two. I just wouldn't go *over* what the stock wheels are, which can happen very easily when increasing the size like I mentioned.
Heavier wheels are definitely more stable and more comfortable, which is why lighter wheels are for racing.