There's something smarmy about going to nude beaches, in my humble opinion, and most of the people I know, who can't wait to get naked in public are plain GROSS. Is it just me noticing this? It's like every paunchy, splotchy, over-moled, bean-poley scoriasis patient is high-tailing it to the nude beaches, whenever they get some time off.
Just about the best swimming experience you'll ever have is in a creek, pond or some small and relatively tame inland water source.
And yes - the nakeder, the better.
I used to swim the creek in the woods behind our school, but it wasn't really about getting naked or 'skinny-dipping'. We only had some forty-five minutes, never knew what depth or tempature the water would be; if the creek looked okay, you shucked off your clothes and waded in. Going naked was also the only way to keep from having a wet stain on my pants, when I got back to class and avoiding pee jokes.
Or maybe, you're out dayhiking, when you find some secluded creek - my favorite, because they're so alive, and you can see right through to the bottom. Scout the woods a little first, if you're swimming with no trunks. If you're modest (like me), wear gym shorts for underwear and be the first one in. [Hey! Where'd these come from? Last one in's your mama!]
One thing I agree with the nudists on...
Swimming in your underpants is for little boys and sissies, and will likely get you ridiculed mercilessly - deservedly so, I'd say. If that's all you're good for, you may as well keep hiking.