Not normal.
I plumbed my secondary fuel filter post OEM and have never lost the prime on it in 20K+ miles. It sounds like we have the same system from Greg Landuyt.
I think I would suspect the large o-ring in the OEM filter, since that seems to be the weakest link in the system everytime I change filters. It has helped to remove the right fender well liner because the access is so much better. About a year ago I replaced the standard plastic anchors on the well liner with blind nuts (I guess that is what you call them - shaped like a squat "C" and they pinch over the sheet metal with a threaded nut on the side away from the liner) so now it takes about 5 minutes to pull the liner.
With your filter pre-OEM, there is a chance it will load up much faster if you have much crud in your fuel - it is going to catch everything. With things post-OEM, I have found 10K to be a good interval for the OEM and 20K for the finer filter. Last change at those intervals both filters looked about equally loaded, but with room to spare for more junk.
2003 K3500 Crew Cab, Banks Stinger Pkg, Exhaust Brake & Exhaust, OilGuard Bypass filter, Lube Specialist Secondary Fuel Filter, Allison Deep Pan, TTT Mirrors, B&W Turnover Ball and Companion Hitch, Truck Trunk Sliding Box and V-Box, LINE-X, Co-Pilot Live GPS, Hitchhiker 31.5 LKTG 5th Wheel 28' Wells-Cargo Gooseneck Car Hauler