For me, it's possible to:
1) Enjoy the sex and come
2) Enjoy the sex, but not come
3) Not particularly enjoy the sex, but come anyway
4) Not enjoy the sex, and not come
I believe the OP was talking about #3.
That said, even though I'm perfectly fine with not always having an orgasm, I'll admit I have a deep-seated insecurity about the man not climaxing. Even if in reality it has nothing to do with me, my emotions still interpret it as a sign that the man just isn't that into me.
One other thing I'll mention is that although I can come from less-than-optimal sex, I find that when I do, my orgasm is generally less-than-optimal as well; it's less intense, less prolonged...kind of like my body's just getting it over with so I'll stop stimulating it already and move on to something more interesting.