Kydex CANNOT Scratch a knife PERIOD. Kydex is softer than steel. Kydex sheaths will make a mark on a polished steel blade but it is NOT a scratch.
Kydex is seen by those who never work with it as extremely simple so the world is filled with self anointed experts who know everything about it and how it works. I've made tens of thousands of kydex sheaths so I speak from first hand knowledge and experience.
As for grit in the sheath, the biggest part of the problem comes from cleaning as you know, but you cannot forget tape residue left inside the sheath from the molding process. Heat breaks down the adhesive and it oozes out onto the inside for the sheath.
Sanding grit is a big issue when you make kydex sheaths; the rule in my shop is to blow out the sheath EVERY TIME a knife is test fitted. We also wipe out the inside of the sheath before riveting with alcohol to remove any tape residue that will grab any grit falling inside the sheath.
We also clean the knife, put it far away from the grinding and cleaning area and religiously wipe it down before every test fitting, again to keep grit out of the sheath. When I started actually cleaning the knife as well as the sheath much of the problem disappeared.
I also leave a lot of room inside the sheath for the knife, using as many as four layers of tape in areas that will be small when the sheath is complete. This leaves room for the crap that falls inside to be away from the blade in use. A single layer at the point and the sheath will still seem to lockup "snug."
One last consideration, Kydex sheaths are ideal for USING KNIVES. Knives in use will show wear so the rub marks left by kydex aren't a realistic concern. If the knife is for show, it either stays out of the sheath or should be sheathed in leather, although even leather will "mark" a polished blade.
The proper measure for a kydex sheath is simple, locks up tight enough to hold the knife in its intended environment and releases the knife smoothly and easily. Beyond that you're looking for headaches.