This is a tricky one, because among all these groups (gay men, bi men and women) there are people who care a lot about cock size and people who aren't that bothered.
However I've been suprised at how many women this is a big issue for, and have openly said that a small cock wouldn't satisfy them. Whereas for me as a gay man who's mostly a 'top', I don't mind at all if the other guy has a small cock; in fact I kind of have a thing for them! And I have friends who are mostly 'bottoms' who point out that doctors can reach the prostate gland with a finger, so any penis of 3" or more should be able to do it and are likewise not that bothered. And, as I'm only too aware, girth is nothing but an inconvenience during anal sex!
As far as I'm aware there's no real statistical evidence on this, but one thing I'm struck by, as a small-dick-fetishist, is there's a fair number of sites for gay guys who are into guys with small dicks, cock comparisons and so on, whereas all the similar straight sites I've come across have invariably been 'small penis humiliation' with women taunting guys for not being big enough. There are gay sites like this of course, but there are also 'I like small cocks'-style gay sites, whereas I've seen no hetero equivalent.
But, having said all that, I think there's much more difference within these groups (gay and bi men, women) than between them!