I believe that what you're wearing and how you carry yourself is of greater importance the higher up the ladder you go. Parents of six year old players want to watch their kids play and don't pay a great deal of attention to how an umpire is dressed. On the other end of the spectrum, at the D1 NCAA level an umpire's appearance is the first impression of how serious the umpire is about his craft (not to mention supervisors' requirements).
Of course, I've always told young umpires, "If you can't BE good, you can at least LOOK good!"
JJ