My theory is that there's also a dimension of patriarchy to it, as counter-intuitive as that might sound. Generally speaking, the US approach to sexuality (which I think is kinda the point of reference here) is focused on the penis: it's what gives pleasure and receives it. And the focus on cock means that everyone's judged, in some way, on their relationship to it: straight men have it; gay men want it; straight women need it; lesbians are a category error because they "reject" it.
Bisexuals of all gender persuasions also pose an issue. The stereotype is that bi women might fuck around with other women a bit, but ultimately are gonna come back to men always—or that their attraction to women is just in the service of pleasing men. And bi men are "contaminated" by their desire for cock—because to desire cock means to override all other pleasures and desires, so aren't we all just actually gay?
Ultimately, bi women are sort of "accepted" more than bi men if only b/c they can still serve as useful objects of sexual desire; whereas we bi men aren't useful in that regard.