It's not just federal court rulings that have decided there should be no religious test for public office.
It's right there unambiguously in the Constitution, Article 6, Section 3.
"The Senators and Representatives before mentioned, and the Members of the several State Legislatures, and all executive and judicial Officers, both of the United States and of the several States, shall be bound by Oath or Affirmation, to support this Constitution; but
no religious test shall ever be required as a qualification to any office or public trust under the United States."
It's great because
in the same sentence we're told that not only is there no religious test, but also that members of the legislative, executive and judicial offices of the states have to affirm the Constitution. Take that, you, pro-religious-test, people, in state offices...
No religious test clause - Wikipedia, the free encyclopedia