Okay, we need to decolonize this idea.
Being "allowed" to work, or more accurately being forced to labor for free, inside the slavers home was not a privilege. You were still being enslaved, and raped, and assaulted, like other slaves.
There might have been some nominal comforts like shade from direct sunlight in the non-air conditioned home, but being beaten or pushed into ovens for burning food, offending a guest, or embarrassing your slaver, or having to clean out the shit of your abusers is not the flex we make it out to be.