Jizz,
Coming from someone who has self-confidence issues himself, let me tell you my suggestion: first and foremost, learn to enjoy your own company. I am about the furthest thing from "perfection" in the physical sense that you'll find. And, the past two years have not been what one would recognize as "wonderful", in terms of family life, etc.
HOWEVER, I perservere. Dad always used to say, in a bastardized Latin, "Illegitimi non carborundum", which he would translate into: "Don't let the bastards wear you down." I was certainly nowhere near popular in highschool and only marginally moreso in college. But, I had a wise advisor during sophomore year who told me that my problem was that I was more mature than my peers and all I had to do was wait until they "caught up" to me. And, you know what, now that I'm 36, I realize how correct he was.
A year ago last May, I was in Europe for holiday. Due to a last-minute change of plans, I had two free days, so I popped round to Copenhagen and visited with a high-school classmate of mine. He was in the "Rebelliously Popular" group back then. Now, he's married with three kids. And while we didn't spend more than 15 minutes together as mates over the three years we attended school at the same time, we went to dinner with wife and then she left us to our own devices. We rolled in about 4am after a night on the town, more ill from having laughed so hard for nearly six hours straight than anything else. The crowing glory was when he told me, "You know, I've been living in Copenhagen for four years now. Do you know how many of [our classmates] have come rolling through Europe and would all call, but not one of them could make the time to come see me? Until you did."
Even at my darkest moments (and there have been plenty, believe me), the one thing that kept me going was a line from "Midnight in the Garden of Good and Evil" where John Cusack's character asks Kevin Spacey's character, after the scandal, if he's going to stay in Savannah. Kevin's response was, "Of course. Me sticking around pisses off all the right people."
I know it sounds like drivel, but hang in there. It may get worse at times, but it will get much better at others and you don't want to miss those!