Donk: I have experienced this and I think there are a few factors that go into it. I agree with bliss that part of it is that you are used to seeing your own penis.
Also, I have read several times (usually in magazine advice columns responding to guys who are worried that their penises are small) that there is an optical illusion that makes angle at which you see your penis when you look down at it makes it appear smaller to you than it appears to someone viewing it from the side. My flaccid penis always gets looks in the locker room. This doesn't surprise me when I know it is hanging in a relaxed, longer state or if it is semi-erect. But sometimes it is in a more shrunken state--from cold or whatever--I will glance down at it and think, OK, it still looks like a decent sized penis but it doesn't really look like anything special at this moment. But it still gets looks. Then I will catch a glimpse of it in the mirror from the side and realize that it still must look pretty impressive to guys of more average endowment. Similar thing sometimes when I am at home with an erection and I stand in front of a mirror. That's when it hits home--yeah, I can understand how an average sized guy would be blown away if he compared his erection to mine.
I think another factor is curvature. Most penises are not perfectly straight. Mine has a significant downward curve. When I look at 11" on a ruler, it seems impossibly long for a penis even to me. If I lay the ruler on top of my penis, the head will curve away from the ruler and it won't hit 11". Yet when I lay a cloth tape measure along my erect penis, sure enough the actual length of flesh from groin to hole comes right up to the 11" mark. Often if I just mention my size to someone who hasn't seen it I will get a reaction like "I'm looking at 11" on a ruler and I just can't believe a penis could be that long." I think they are picturing a straight perfect cylinder, like a big Pringle's can sticking out of my groin. Picturing a column of flesh in that size and shape, I can see why anyone would be skeptical. But most penises, like other body parts, have a more natural, irregular shape. Fortunately, I think the natural shape of my penis also makes it easier to fit into a woman than a Pringle's can would be.