The key to crystal clear water is "sticky" filtration medium, generally old mature sponges etc that grab tiny particles via sticky bacterial infestation etc. Your gravel bed should be doing that, unless it's very new.
Personally, I wouldn't worry about it unless your fish do.