Xanthid Crab?

Hey guys, so I have a big dead coral skeleton as a center piece in my tank and I found a full grown, I think xanthid, crab that is a hitchhiker and is picking algae off of the inside of the skeleton. I've been watching him for a few days and he hasnt touched anything besides algae. He is hairy with slightly black tipped claws that are slightly curved. I'm having a hard time removing him as he is way down in the skeleton and I can't seem to get a pic of him. Should I really try to remove him? He hasn't done anything bad to my live brown button polyps that are on that skeleton and by the size of him, hes been there for a long time. What should I do? Thanks, I'll keep trying to get pics.