The saffron party formed the government in alliance with the newly formed Jannayak Janta Party (JJP) led by Dushyant Chautala, 31, who took oath as Khattar's Deputy Chief Minister.
Governor Satyadeo Narain Arya administered the oath.
Barring Khattar and Dushyant, no Cabinet member took the oath in the ceremony in Raj Bhawan that lasted less than 10 minutes.
Khattar, 65, will be heading the second Bharatiya Janata Party (BJP) government of the state, which was formed on November 1, 1966.
A low-profile organisational man with an active RSS background, Khattar was chosen by the newly elected BJP legislators as their leader at a meeting here on Saturday.
"My government will be transparent," Khattar told the media ahead of taking the oath.
Five-time Punjab Chief Minister Parkash Singh Badal, whose party Shriomani Akali Dal (SAD) had an alliance with BJP's rival Indian National Lok Dal (INLD) during the assembly elections despite being an NDA partner, attended the ceremony along with son and Member of Parliament Sukhbir Badal.
Congress leader and two-time Chief Minister Bhupinder Hooda, who attended the ceremony, described the BJP-JJP alliance opportunist, unholy and against the mandate.
Dushyant's father Ajay Chautala, who was released on furlough from the Tihar Jail, attended the oath-taking ceremony along with wife and second-time legislator Naina Chautala.
"Dushyant asked me if he should go with the BJP or the Congress. I told him, whatever the circumstances, we shall never go with the Congress. I gave him the go-ahead to go with the BJP," Ajay Chautala said, adding "it will be beneficial for the state."
BJP working President J. P. Nadda, Himachal Pradesh Chief Minister Jai Ram Thakur and Uttarakhand Chief Minister Trivendra Singh Rawat, among others, were present during the swearing in ceremony.
Earlier, the BJP, which won 40 seats and was six short of a majority in the 90-member Assembly, announced an alliance with the JJP led by Dushyant Chautala, a great grandson of former Deputy Prime Minister Devi Lal.
Dushyant Chautala parted ways with the Indian National Lok Dal (INLD) in December 2018 after a bitter vertical split in the party and the Chautala family and formed the JJP.
( With inputs from IANS )