I have always wondered about this as well. According to the women I have been with (not all that many, but...) most guys tend to wilt pretty quickly. I have always stayed firm at least and have usually been able to go for another round. Never knew it was unusual.
Age has tempered that a little bit. I still stay hard for a few minutes (5?) but the 2nd go-round is a bit iffy sometimes.