A few thoughts:
1) The average length of six inches, which most young men grew up hearing, comes from the grpound-breaking Kinsey Report (Sexual Behavior in the Human Male, 1948, Alfeed C. Kinsey). Most of his findings have been confirmed, but the penis size data was flawed because it was based on self-reported measurements. Subsequent studies relying upon more objective methods found an average length ranging between 5 and 5-3/4 inches and a girth somewhere around five inches. A lot of neurotic feelings of inadequacy probably stemmed from the wide-spread acceptance of six inches as average.
2) The reason why otherwise average-sized penises are called small is probably simply due to the fact that they're not big. After a woman's felt a big cock, especially one of great girth, she may feel anything less is small, and anything less than that really small. I experienced that when a girl, seeing my cock for the first time, blurted out scornfully that I wasn't so big; her uncle, who she'd once glimpsed with morning wood, was lots larger.
3) Watching porn is probably a contributing factor. After seeing mostly big and gigantic cocks it's easy to assume that's the norm. I'm surprised at how many guys on the internet call perfectly average-sized cocks small. Obviously they've never taken a look around a locker room.