gobies and vacuuming sand?

I am not at the point of adding fish yet and a goby is on my list, but not at the top. I am keeping sand for aesthetic purposes and plan to vacuum it with each water change. But if I have a goby that makes a burrow (or jawfish) will I be destroying his home? Should I avoid vacuuming near the burrow or will they just build it again after I finish? Just thinking ahead here.