OK, one man's perspective. I have suffered with very similar things myself over time. I would say that it is performance anxiety.
First of all, Sassy, there is nothing wrong with you. Don't think that for a minute. And if it isn't too late for this, try not to let him know that you are wondering if you are somehow inadequate. Here's why...
He was a virgin for a long time (as was I, until I was about 22). He has been thinking about his first time with a woman for the last 10 years or so. He has all kinds of hopes and expectations about what it will be like. He doesn't realize the enormous pressure he has put on himself for sex to be magically wonderful. As good as it can be, it is surely different (and probably not as good) as what he has imagined.
So, he penetrates you, and there is suddenly a disconnect between what he has believed sex would be like for 10 years, and what it is really like. He almost certainly believes you to be normal (maybe even perfect!) so the problem must be with him. Self-doubt rears its horrible ugly head. He starts to worry that he is doing it "wrong," that he is not pleasing you, not pleasing himself. He feels like maybe his erection isn't as hard as it was at first... and once he worries that he is losing his erection, it is as good as gone. Now he feels even WORSE... he is with a beautiful woman, and he can't even get it up. "What the fuck is WRONG with me..." He is probably close to tears inside. He feels like a miserable failure. He has let you down. He has let himself down. He has been waiting and wanting this day for 10 years, and it just sucks. It just fucking sucks.
That is what performance anxiety is.
My advice to you... be confident in yourself. Don't display your disappointment. Assure him that everything is OK. Assure him that you love being with him, that it is good now and will only get better as you become accustomed to each other. I would recommend that you NOT make a positive comment if and when he is successful, because that will only put the pressure on him for a repeat performance. Let the success just be one of any variety of good pleasures that come from being together.
This is all about gently building his confidence, letting him feel secure enough to accept that sometimes he'll be great and sometimes he'll be crap, but that it is entirely OK.
I hope this makes sense, and I hope it helps...