It is for the first time the state is seeing non-Congress government at the helm for the second consecutive term.
Second-time legislator, Khattar, who belongs to the Punjabi community in Jat-dominated state's politics, on Sunday was sworn in as Chief Minister.
However, his party failed to cross the halfway mark in the 90-member Assembly.
The saffron party, which won 40 seats and was six short of a majority in the 90-member Assembly, formed the government in alliance with the newly formed Jannayak Janta Party (JJP) led by Dushyant Chautala, who took oath as Khattar's deputy.
A grounded organisational man known for his simple lifestyle, the 65-year-old Khattar was chosen by the newly elected BJP legislators as their leader at a meeting in Chandigarh on Saturday.
Born in Rohtak district, Khattar, who aimed to become a doctor, had contested his first assembly elections in 2014 from Karnal where he won by a margin of 63,736 votes.
This time he retained his seat by defeating his nearest rival and Congress candidate Tarlochan Singh by 45,188 votes.
Before Khattar, Bhajan Lal was the last non-Jat Chief Minister of Haryana from 1991 to 1996.
Khattar believes in the principles of creating a 'Ram Rajya', meaning an honest government with zero tolerance for corruption.
BJP working President J. P. Nadda told the political culture of the state has been drastically changed in BJP's first stint.
"It was corruption-free, development oriented and transparent government," Nadda said.
As per the poll share in the Lok Sabha elections held in April-May under Khattar's leadership, the BJP, which won all the 10 seats in the state, got the lead in 79 Assembly seats, the Congress in 10 and the Jannayak Janata Party (JJP) in one seat.
The BJP, which had won 47 seats in the last elections and formed the government in the state for the first time, was reduced to 40 this time, well below the 75 plus target.
BJP's outgoing ministers - Captain Abhimanyu, O. P. Dhankar, Ram Bilas Sharma, Kavita Jain, Krishan Lal Panwar, Manish Grover, Karan Dev Kamboj and Krishan Kumar Bedi - all ended up with humiliating defeats.
Political observers believe the challenges ahead of Khattar in his second term is - rising unemployment, law and order, mounting debt and poor remunerative crop prices.
In his first stint, Khattar's name had virtually become synonymous with inept handling of tough situations - be it the violent Jat agitation that left 30 people dead, or violence during incidents related to self-styled godmen - Rampal and Dera Sacha Sauda chief Gurmeet Ram Rahim Singh.
Even the Punjab and Haryana High Court time and again came down heavily on the Khattar government for mishandling the violence during the Jat agitation too.
But his major achievements were to deliver a corruption-free government and transparency in recruitment for government jobs.
Focusing on farmers, labourers, youth and industrialists, the saffron party manifesto pledges to give an interest-free crop loan of up to Rs 3 lakh to farmers and promises to complete a target of Rs 5,000 crore interest and penalty waiver on crop loans taken from cooperative banks.
The BJP has also promised to give collateral-free loan of up to Rs 3 lakh to people belonging to the Scheduled Castes and free education to two girls from families whose total annual income is less than Rs 1.8 lakh.
(Vishal Gulati can be contacted at vishal.g@.in)
( With inputs from IANS )