If women were as fixated on size as men (in general) there would not be so many threads on topic. Men would be able to get the specific feedback they seek.
I think it is natural curiousity to know how one stacks up and if done in healthy manner those that are not large or even average then can move on to what they are good at and to focus on those things. Too many guys get stuck for a while on the size thing simply because though very much appreciated by women when their lover has it the size is rarely in top-5 or facsimile and they don't have the specific memories like a guy and his focus would create.
Being an old guy that has spent most of work days in predominantly female work areas I have learned my fair share from being the "safe guy" and a good listener.
A women with a lover that is on high side of average would know and be able to tell current lover he was bigger than prior IF said prior was on low side of average (Redbear52 post) given there is a relatively wide range in the middle ground. It would be rare for even a women of high sex drive and courage to act on it to tell a guy in that average range exactly where he stacks up though for her total history. She will be able to, if so inclined to spell it out, tell you a general breakdown along lines of "been with twenty guys, a few modest, one very small, a couple meaty ones and one guy that was a freak of nature in size and the rest about like you".
So, if you want to favorably match up for your lover and you are in the broad space of average hope you follow a guy that is very tiny like the porn stars Ed Powers and Randy West from the old days. Those guys made a living in porn yet have incredibly tiny penises. She will have that general range of experience in her head yet think your average penis is larger than average.
Whatver you have use it well and she, again in general, will be very happy with it. Why? Because it is yours and if you are loving towards her then it will be all hers - ha ha.
Other than hoping that you don't follow some stud - not far enough back to be clumped into her hazy memory banks, the only bad size is the ones she won't tell you about. If she is always diverting away from your questions on this she is hiding something. I have heard enough women talk about current guy that is less than and though she is in love and all that there will always be that guy she remembers. If she tells you about him/them then they are in memory banks along with other life experiences. If she does the whole "I don't recall" (again, can speak in general terms like "mostly about same size except a few more modest than you a couple bigger and one too big sometimes for how he jackhammered") then she is settling for what you have. That is fine unless you want to someday be her one and only - you can achieve second but only status and if that is fine for you then go fot it.
I find it fascinating what I have heard women talk about with just the girls around (yup, that includes me - joke on me obviously) yet have never come across a thread with similar straight forward feedback even on this site.
. But I agree with you about those who are on the borderline. If there is something else remarkable about the sex, or the penis itself, it can probably push you over into the 'remembered as large' category. 
