Wow thanks for all the replies!
I guess it is more of an issue with me, than with women; as most of the women here seem to have backed up what my ex-gf also said, that erect size is what really matters. And also I guess 3 or 4" isn't THAT small when flaccid, so thanks all! I understand for some women that flaccid size does play some role, but the responses I got made me feel a lot better about the issue.
Its probably more of my own insecurities with my body anyway. Its like I was programmed and cannot un-program myself to believe that a mans masculinity and manhood are not directly connected to his dick.
For so long when I was growing up, and waiting and waiting for my dick to be "as big as Dads"(no I wasn't molested perverts I lived in a house with the man for 25 years!
I always thought that men measured other men, and women measured men by the size of their dicks. Anyway, I guess that I should just learn to be happy about myself the way I am, which is what I strive for every day, to be the best possible version of myself.
Thanks again peeps!