And it all goes back to Lawrence v. Texas:

Jonathan Turley is probably not the most popular man right now with supporters of same-sex marriage. The George Washington University law professor has filed a suit challenging the constitutionality of Utah's anti-polygamy laws — and his argument is based on a landmark 2003 Supreme Court gay rights decision. That's not good news in the view of most gay rights supporters, who don't want their cause linked to that of polygamists any more than they want to see parallels drawn with people who engage in incest, bestiality and other taboo sexual practices.

... The 2003 gay rights case, Lawrence vs. Texas, was also a criminal matter unrelated to same-sex marriage. The court overturned the conviction of two men found to have violated a state law against same-sex sodomy. But in reaching that conclusion, Justice Anthony M. Kennedy offered a paean to intimate relationships defined by sexuality that easily can be transferred to the context of same-sex marriage, and potentially to polygamous marriages as well.

... like Lawrence, a ruling sympathetic to unconventional sexual behavior could plant the seeds of a future campaign for full marriage equality. In that case, governments would have to prove that it's rational to limit marriage to two individuals, homosexual or heterosexual. That might seem obvious, but so, at one time, did the argument that marriage should be confined to opposite-sex couples. --Los Angeles Times