Sorry I am not familiar with your set up or husbandry. Care to share what you are working with and your routine ?
Do you siphon food up after feedings? How often do you change water etc?
Were the sponge filters Dirty? Have you seen any issues with your fish.
Sounds to me off the bat like you just have a clean system and seems like you clean your stuff good and often so they shouldn't be too bad especially w the prefilters. You'd be surprised how much that helps.
Most people wait months to clean canisters and that's when the grossness comes in but your cleaning 3-4 weeks so looks like you are doing a good job