From a Dom/Sub relationship it is understandable as it can reflect that Dominant, Authorative relationship between the two. I've been called Daddy several times. Not all of them during sex. It's typically used by the gay men I know to describe my personality and stature when not used during sex.
I'm also aware that it has become a popular trope in porn. I'm sure you've seen it before. Two people roughly 5 years apart in age, both under 30. At some point whoever is getting fucked says either "Fuck Me Daddy" or "Ay Papi". I might be alone here, but if you're going to say something during sex, I'd like you to mean it. If I start hearing lines you've heard in porn, I take it to mean that I'm not doing a good job. So I obviously just need to stop. Any I typically do. Get dressed, and exit stage left.