I supposed the answer depends on your washing method and choice of products. I will say that "Yes" there could be problems with washing every day. Some car wash shampoos can be harsh and will dry out trim and seals over time. Washing daily can accelerate this process. In addition, surface marring is often created during the washing and drying process. Every time you swipe a towel or mitt across the surface, you run the risk of inducing a swirl or scratch.
Some food for thought -
Consider using products known for attracting less dust. If the dust only seems to accumulate on the back portion of the car, then concentrate on daily maintenance of that particular section. Leave the more dust resistance sections alone for awhile. If you insist upon washing frequently, it will be essential to choose a gentle shampoo with high lubricity. You will also need high quality washing and drying implements.