The cuckold lifestyle is not specific to any stereotypes of human physical traits. Being "cuckolded" has nothing to do with penis size. It's more so about loss, dominance, and humiliation, followed by acceptance.
A cuckold loses his woman to another man who is "better" in whatever fashion (again, the other man doesn't necessarily have to have a bigger dick although it is the general acceptance) and the cuckold is humiliated for the loss however this is where the cuckold thrives. He loves the fact that his woman is taken away from him, that he's humiliated for it, that she is treated in a better way than he can provide, and is sexually charged from the entirety of it all.
The thing that most men, if not all men whom are not into this lifestyle, cannot understand is that a cuckold fully accepts this state of situation and looks forward to the next encounter and the next and so forth with eager anticipation and may even facilitate future events by hunting down other men (or "bulls") for his girlfriend/wife. It goes against the masculine narrative to mate-guard your woman against other men.
The one key aspect of all this and probably the most important is that his woman never goes away from the relationship. She's taken for the moment, the night, the planned event, the whatever, however when it's all over, she returns back to the cuckold and the cycle repeats. She enjoys being in the dynamics of the relationship as much as he does. That's the beauty of this lifestyle; it just works for both parties therefore it's imperative that these two people match up as much as possible. Perfect matchups do exist and they live very happy lives unbeknownst to their neighbors. You pass by these couples daily and wouldn't ever consider the possibility.
@matt1027 you are the outlier but certainly not the exception or the oddball. I've met plenty of good looking, masculine, straight men (mostly married) whom are totally into this lifestyle. Some of them have big dicks and fuck incredibly well however they love to feel the emotion of being cuckolded by their wives. They all have their own fetishes and kinks in this area... some of them prefer their wives with black men, some prefer their wives being with more than one men at once, some of them prefer their wives do it with them in the same room, some of them prefer their wives go out on dates with other men and return to tell them about it (or text them during or bring back a creampie, etc), and the list goes on and on and on...
Lastly, don't mistaken this for swingers. There are lotsa men whom enjoy watching their girlfriends/wives fuck other men and have no interest in the cuckold lifestyle. I'm one of them. My wife and I both actively fuck other people when we can and whom we can get along well with. I have no issues with her fucking other men as long as she's open about it with me, and vice-versa. I fuck other women all the time and have no issues telling her when it happens. I have no interest in being a cuckold. I don't "share" my wife, I simply allow her to fuck other people because it doesn't bother me and she does the same for me. I've also been the "bull" many times over and I can't really say I enjoy it or not. To me, it's just good sex and the cuckold husband gets a thrill out of it so it's win-win, in my opinion.
As to why you are this way... well, nobody here is going to be able to tell you. If you're truly curious, I would encourage you to seek out a skilled sex therapist that you can decompartmentalize your inner demons to try and sort out where this all originated from. It's in there somewhere in the deep recesses of your mind and well-being.
Good luck and be careful whom you share this with. Some people will embrace it with you while others will shun you for it (even your own girlfriend/wife, sadly).