Always give your towels (polky or cotton, microfiber or larger) a wash before use. All textiles have a finish in them that aids in the weaving or knitting process. All textiles are also chemically finished once woven. Then they are handled in the factory and when in the cutting and sewing rooms people handle them with not so clean hands and they pass through oily sewing machines. There is no way around this so you should always clean first.
By the way, this also applies to bath towels used in the house, always wash first in hot water.