Leviticus 16:20-22

Then a man specially chosen for the task will drive the goat into the wilderness.
As the goat goes into the wilderness, it will carry all the people's sins upon itself into a desolate land.