I do this all the time, people form their opinions quite swiftly on appearances (sad but true) and clothing can definitely help shape that. I generally like to dress quite smart and like the effect it has on people, but at the same time I like dressing in dull hoodies and sweatshirts if I'm popping out to the shops and don't really want any attention.
I work alot with people in suits in high-powered jobs and I find sometimes dressing casually around them can have a weird effect too. It's as though they perceive I must have some specialist skill or expertise that allows me to come in to the office wearing jeans and not bothering to tuck my shirt in. It's a different story when clients are involved though, and I'll generally always dress "eccentrically smart" as it seems known in my job. It's kind of expected that I don't look like a businessman, in a way.