Despite routine abuse, the concept of "equality" is not a magical imprimatur which can be stamped on every argument.
Your argument is exactly parallel to one of the arguments offered in Loving vs. Virginia, which was rejected by the Supreme Court as not legally valid. The defendants in Loving argued that the law was "truly" equal, since a white person could marry someone of their own race, and so could a black person.
From the majority ruling in that case:
Under our Constitution, the freedom to marry, or not marry, a person of another race resides with the individual and cannot be infringed by the State.
And so, despite the "equality" of the marriage rights in Virginia at the time, the law was found to be an infringement of individual freedom.