Cheika announced on Sunday night that he will be leaving the Wallabies after five years as coach, the day after his side was thrashed 40-16 in a much hyped make-or-break game against historical rival England, reports Xinhua news agency.
Previously Cheika had said that he would leave the position if Australia failed to win this year's World Cup.
"I'm the type of man who is always going to back what he says and I knew from the final whistle but I just wanted to give it that little bit time to cool down, talk to my people and then make it clear," he told reporters in Japan.
With his current contract finishing at the end of the year, Cheika said that he will not be seeking to renew his tenure as coach.
Despite reaching the World Cup final in his very first year as coach back in 2015, the team's performance under Cheika has steadily declined, leading some to criticize his leadership.
Overall, from 68 games in charge, Cheika walks away with a record of 34 wins, 32 losses and two draws.
In an open letter to fans, Rugby Australia chief executive Raelene Castle thanked Cheika for his "dedication and service" to the game, and said that the search for a new coach is already underway.
( With inputs from IANS )