I think one of the biggest problems is that when you have a culture where it isn't common to be naked in non-sexual situations, that the main point of reference that many men have of what a male body "should" look like, is on underwear/swimwear adverts, locker rooms and in porn. In those sort of environments, you tend to see very fit and athletic bodies. Additionally, gay bars, magazines and pornography will often place similar body types on a pedestal. Those sorts of bodies are very difficult to achieve for many.
To be honest, I'm surprised at how little culture has been referenced in this thread. I've definitely seen quite a big difference between Germany and the UK on this one, I'm from the latter and live in the former. Non-sexual nudity is certainly more common here (the UK tends to be more reserved on this front, but probably still more relaxed than other countries), saunas are popular here and there are parts of parks that are FKK (Freie Körper Kultur - free body culture, literally) friendly as well. If you go out to the east, there's even more of a nudist tradition, because it was one of the only freedoms they had during the GDR dictatorship.