If you're thin and don't gain weight, who cares how much you eat? If you eat very little but are overweight you'll be judged and viewed differently by your size, not your consumption...and if you pig out, gain weight AND are overweight, you'll really take a negative hit as far as preception goes!
You hit the nail on the head. This describes my mother, who eats very little but manages to carry excess weight. She is active even though she has a disability, she eats small meals and maintains a diabetic diet. Despite all of this, she has been viewed differently because of the extra weight she carries.