I think that most people in every culture feel this to some degree. Some people more than others, of course. I have only been with my husband, but my husband has been with a number of women before me. It used to bother me a lot.
For the first two years of our marriage I was the most jealous bitch imaginable, checking the time stamped on his train tickets, seeing if he took a comb with him to work (to look good for the women he was having affairs with at the office,) smelling him for perfume when I met him at the door, insisting we change tables if our waitress gave him too much attention, etc.. Most of this behaviour came from my insecurity that he knew how bad I was in bed since he had been with so many other women.
My best friend listened to me rant at him one day at her house, and told me if I kept it up he was going to do exactly what I feared most: stop loving me and start loving someone else. I really tried to cool it down after that, but my husband was really wary about anything having to do with women; didn't like going to films with me (sex scenes used to set me off), didn't want to attend my friends birthday parties, didn't want to go to night clubs. I realized how much I had damaged the relationship, and saw that I was making my life worse rather than better.
Ultimately what put all of the insecurity to rest was when he told me "I loved you enough to marry you without having had sex with you, those other women in the past... I had sex with them but didn't like them enough to get married, so that should tell you something about how I feel about you." The things that always worried me were that I knew he could have other women easily, because he had, and that I might not measure up to the women in the past. So when he said all that to me, it really dispelled a lot of my irrational fears. I also felt really ashamed. He married me because he thought I was the best woman he knew, and then for two years I was acting like a complete idiot. I felt like I had really let him down.
Since then, my trust in him has grown and grown, and the thoughts of his past women don't really bother me. Occasionally they do pop up, but then I just tell myself "he dumped that bitch 'cause I am too damned sexy" or something silly like that
We now have another couple that we play with sometimes, so I think I have really conquered my insecurities. However, I still have limits; the other couple is pretty deep sub. and I get to switch dom. when we play with them. That way, I get to keep things within my comfort zone. I'm very happy now