There have been a lot of surveys where women place a huge amount of importance on a man being taller than they are. I've dated a couple very tall women (top 0.5% of women's height) and they were 0.5-1" shorter than me. If I was just half an inch shorter than them, they probably wouldn't have been interested.
Since men are overall ~4-5" taller than women, only someone very short for a man, or very tall for a woman, or some match in between will make the pairing uneven. Some guys who are smaller but not necessarily 'short' (e.g. 5'7" or so) are more likely to see taller women, and since they have a harder time getting those taller women will just want them more. In truth, at 5'7" only about 1 in 5 women would be taller, so you'd still have a good sized dating pool, but for people with insecurities a little rejection tends to get blown way out of proportion.
It's really the same for really tall girls as it is for short guys, since most guys don't want a woman taller than them either. For the exceptionally tall (for a woman) that I've dated, their height really excluded them from just under 80% of all men. Since they were pretty unique girls already who had a tough enough time making a connection, losing 4 out of 5 potential suitors really sucks. One had huge hangups about not feeling feminine enough, when in reality she was the most womanly woman I've ever been with.
Here are some percentile graphs you can check to see how much overlap their is at different heights:
Women:
Average womens height chart
Men:
Mens average height chart
However, I think there can be a difference between the results of what people say they want, and what they actually end up doing. I know several women who find tall, fit men attractive, but are with chunky short guys who they absolutely love to death. If it's a real, primal attraction and a true emotional + intellectual connection, things like this become less important. When it's a less important relationship it matters a lot more, but if it's a less important relationship then it's not worth getting all bent out of shape about since it doesn't matter that much.