As far as compliments directed at me, it is all a matter of context. I don't mind comments of admiration from other guys in any setting as long as they are non-sexual such as it relates to my muscles, maybe the shirt I'm wearing (sometimes adorned with a geeky or humorous image or saying) or the fact a lot of people feel I don't look my age, to name just a few.
Mild sexual compliments or borderline flirting are a different matter dependent upon the setting. Somewhere that is predominately represented by gay or bi men, such as online forums like LPSG or the times I was contracted to work at LGBT clubs, I feel it is to be expected and never bothered me. In fact, I frequently took them as genuine compliments and appreciated those that gave them as long as they had the understanding that appreciation didn't equal interest and such accolades were not going to change my exclusive interest in women. However outside of those venues, say in a locker, rest room or other public/work setting, sexual compliments from other men I would find inappropriate and would creep me out.
As far as me giving compliments, while I admittedly am not the most generous when it comes to strangers or semi-strangers, but I 'd have no issues giving non-sexual comments to other guys in a public or work setting on something I feel is worth admiration such as their muscles or a hobby or task they are good at. Being straight, I of course have no need to comment to another guy on any sexual features.
For women outside of a sexual setting, such as a swing club or party, I pretty much stick to the same non-sexual criteria, but it can go either way, with me being a little more complimentary or a little less versus with a guy, depending on the setting and my personal judgment that the woman might or might not object. Some of the comments might have been framed in a way that were mildly flirty, but could still be seen as non-sexual in case they were taken the wrong way such as "I like your dress", "nice hairdo" or expressing disbelief and doubt that she looks her age. I've found a lot of people, but especially women, have the habit of associating an innocuous comment with self-validation and thus complimenting them on something like what they are wearing translates into it's not so much that their clothes (or hair) look good in and of themselves, but it's only because they are wearing them and it makes them feel good, as well as a roundabout way of letting them know you find them attractive.
However, I should point out again, that such comments are dependent on the circumstances and the individual and because of this while I may compliment women more than other men, even those are rare for me. Though I've never felt the frequent need to be even mildly indirect with others, there is just too much that can go wrong nowadays, that ensures it is definitely a once in a blue moon type of thing.
So, unless one can be pretty damn sure someone, whether another man or a woman, will not be offended or find even a mild sexual or flirting compliment inappropriate, I think refrain is the best course of action, especially in a business or public setting. Either stick to totally innocuous comments or just keep it professional and treat everyone with respect as just another person, even if they catch your fancy in some way.