Remember the old aphorism about beauty being in the eye of the beholder? Physical attraction spans all body types. I have some friends who are somewhat attracted to the stereotypical "hot" muscled-up and toned type of guy, go absolutely nuts for a guy with a sizable gut.
The key to being attractive, regardless of body type, is to be comfortable in your own skin, and by extension, to have your body be an accurate reflection of your self-image. When those are in harmony, it shows through in your attitude and in your facial expressions. Some people can pick up on it consciously, but most will pick it up on a subconscious level.