Question about Zooanthid poisoning..

I'm just beginning my reef tank and want to know if I always need to wear gloves while in my tank or just when handling the coral? Thanks