"Taking moral responsibility, I am resigning as the Karnataka Pradesh Congress Committee president," said Rao at the Congress office.
Rao said the results were not according to his expectations and being the KPCC President if he doesn't take the moral responsibility who will?
The Congress leader said he spoke to All India Congress Committee (AICC) General Secretary K.C. Venugopal about the defeat.
Rao said that the Congress thought people will not favour the defectors, yet the BJP won.
"If such mandates come, it will only further encourage the 'operation Kamal'. The operation has been ratified by the people," said Rao, describing the way BJP attracted and fielded the disqualified Congress lawmakers.
The BJP had fielded 11 Congress and three JD-S defectors to wrest the seats from the opposition parties.
"We fought a good election. We fought unitedly. Media also supported us in many ways during the campaign, ultimately we bow our head to democracy. That is beauty of democracy. We accept people's verdict and then take our movement forward," he said.
Rao said he will meet Congress president and submit a detailed report about the party's rout in the bypolls.
Sounding hopeful of the future, Rao said: "We will take whatever directions the party high command gives us and we will build the party".
Denying to give the takeaways for Congress from the electoral drubbing, Rao said he will not analyse the defeat right now.
Earlier, following a humiliating defeat in the by-elections, Siddaramaiah also resigned as the leader of opposition in state Assembly.
( With inputs from IANS )