From what I can tell, the problem is in your second image in the decal.
Adjusting that will resolve the uniformity problem.
At a guess I would suggest the image isn't a proper Normal Map.
Changing the decal type from 'Normal' to 'Bump' 'Displacement' considerably lessens the adverse effect.
Edit: Displacement can produce some pretty interesting cloth effects.
The artifacts point back to problem areas in the image.