Sump Cleaning

I purchased a tank from a guy who had the system running for a while he and everything was healthy in the tank, Ive had the system running for about 6 months and have noticed a build up of what looks like dark brown sand in my sump. There was some there when I bought the tank and the guy said its good and that there are a lot of good organisms that live in it and clean the water and that he never cleaned it out and never had a problem, I have to weekly water changes for the past 6 months and my nitrates won't go down. My TDS meter says 0PPM for my RODI water so I was think that the "brown sand" is detritus build up and that it needs to go. Am I right on thinking this way or am I miss lead. Do you clean your sump and if so how often and how thorough. Thanks in advance for your help