Do you comment on other guys' dicks to them or other people? I've complemented physiques before, but never said "you have a nice dick" to another guy, or talked about a guy's dick to other people. That kind of comment (in my circles at least) would either get you shunned or a punch in the face.
The first commentary about my cock was made to me as a teenager. I found it to be complimentary, and never thought much about it until years later. After many years of guys making comments about my cock, and composing various gracious responses, I've discovered expressing gratitude and acknowledging their cock's attractiveness, also to be liberating. I guess that I would prefer not to associate with a "puncher".