Have you thought about the ultrasonic repellents?
I'm not sure how good they are as I've never used them, but you plug one in in every room in the house and they're supposed to annoy the vermin so much that they leave. I don't know if your puppy would like them though, so that would be something to investigate beforehand.
Cats can be good at keeping them away, but they often bring them in, play with them, get bored and walk away, or the mice sometimes escape under furniture.
Jason's suggestion with the traps is a good one. Let them get used to it first and then snap! Good baits are peanut butter, and chocolate and Snickers bars are meant to be about the best you can get.
You should be able to get little poison containers which the mice can get into but your puppy can't. It's probably worth getting some proper advice from an exterminator.
Seal up any holes or gaps that you can get your finger into and that will help, but it's no guarantee.
Good luck.
A few years ago, I went home for my sister's wedding. I remember waking up one morning with a mouse running over my head. I freaked. It was two days before the wedding and all of the wedding presents were in my room. My other sister and I went through everything, searching for the mouse and checking for damage. We eventually managed to trap it using a badminton racket and the tube my postgrad diploma came in! I did the sort of slightly humane thing and released it outside. Unfortunately, I don't think it really appreciated being launched out the end of the tube at arm's length.