I am a sexual submissive, so I'm into kinky sex and kinky talk more than the average person. I'm very turned on by "that's my girl", "good girl", and, I confess, even "little girl", but I've never been asked to call a lover daddy. This makes me think it's not all that common, because I've had quite a few lovers.
Hmm...if my Master asked me to call him Daddy, I probably would...it produces two feelings in me at the same time. One is revulsion -- as someone said the last thing I want to be reminded of during sex is my father. But the other is arousal (am I the only person here honest or brave enough to admit this?), and I'm probably aroused just because of the taboo (sometimes, breaking taboos, especially being ordered to break them, turns me on in and of itself).
As for the phrase, "Who's your daddy?", our culture has (in the last couple of decades) drastically perverted the original meaning.
That phrase was originally a taunt given from schoolyard kids to other kids -- the meaning being that the target kid's parentage was in doubt. If the kid was of a single mom who had never married, it could be a particularly painful taunt, as often the kid did not know who his or her daddy was, and being a "bastard" was a great source of shame. Also, in small communities, it was a legitimate question, since any man around just might be the daddy in question. So, particularly nasty people would whisper to someone, "Who's your daddy?" as a vicious insult.
How the phrase became something sexual, I do not know. I wouldn't want to be asked it during sex.