Lehigh County, Pennsylvania, has to remove the symbol of the cross from its seal, a federal judge ruled.

The Freedom From Religion Foundation, a secular group, filed the suit on behalf of four citizens who said they’re uncomfortable with the Christian symbol on the county seal.

U.S. District Judge Edward Smith ruled in favor of the plaintiffs, even though he expressed he was not happy to do so.

“While the court does not believe the current state of the law applicable to this case comports with the text of the Establishment Clause, the court is not in a position to reject it,” says the ruling. “The law, as it currently stands, requires that the court rule in favor of the plaintiffs: the inclusion of the cross lacked a secular purpose both when the defendant adopted the seal and when the defendant refused to remove the cross from the seal, and a reasonable observer would perceive the seal as endorsing Christianity.”

The FFRF first threatened to sue the county over the cross in 2014, but officials refused to remove it from the county seal and flag.

Leghigh County first adopted the seal with the cross in 1944 and it appears on flags, buildings and the county website.