There's generally been a trend, since at least the Roman Empire, to associate penis size with masculinity (there have been exceptions. Ancient Greece was one of them.) That's intensified by the general U.S. (and associated Western) culture in which bigger and more are generally always seen as better, which is of course not necessarily true.
That said, although I'm significantly bigger and thicker than average when erect, I don't look at all big when flaccid (Cf. "Grower,") most of the embarrassment I've experienced when nude in a group of people has been self-generated.
I've had friends with extremely small dicks. I know this because either I've seen them (locker room, pool, or in a couple of cases fooling around a bit, or because they've told me about it.) I've never made fun of them, nor been negative toward them about it. I've known (and seen) guys larger and / or thicker than I when erect, as well. I had one friend I used to tease about it, because it made him blush like hell (but one could tell he liked being teased about it.) Else, I didn't comment to them one way or the other, either.
Frankly, so long as one's partner enjoys the sex and doesn't have any complaints, I don't see how it should be a problem. That said, I don't really think that laws restricting speech, are a solution to folks feeling embarrassed about their penis size, any more than I think laws restricting speech regarding one's weight, hair color, &c. are a solution to other such embarrassment. Speaking as a US citizen, there is no constitutional right to not be offended.