I don’t feel that it is a matter of right or wrong. I am slightly larger than you and do not go commando in certain situations since I do not want my dick to be prominently displayed or a topic of conversation. For example, I may stop in at my office while commando on the weekend when no one else is working, but I would not go commando at work during the week or at a weekend conference. I also would avoid going commando with family, formal occasions and anytime that I feel I may alter the professional or personal relationships or perceptions that people have of me. Those things are important to me, so I am more cautious.
The times that I am most conscious of my bulge are those times when I am cycling. I sometimes end up in mixed company or around senior citizens and I just don’t feel comfortable. But, I am also the type that never swears in the presence of women and would never wear an offensive T shirt with the images or words in public. That is just me. I am not conservative, in fact I lean liberal, but I just have some core values that are important to me.
So, I limit my commando time and thoroughly enjoy it when I do it.