The Congress rebels had joined the Bharatiya Janata Party (BJP) on Wednesday.
Goa Chief Minister Pramod Sawant told reporters at Raj Bhavan that Kavlekar, a former leader of opposition, would be appointed Deputy Chief Minister and a notification formally allocating portfolios to the four new inductees could be issued by Monday.
The swearing-in ceremony followed hours after the unceremonious sacking of four non-BJP ministers by Chief Minister Pramod Sawant after a day-long stand-off. During the stand-off, Vijai Sardesai, Vinod Palienkar, Jayesh Salgaocar all ministers from the Goa Forward Party and Rohan Khaunte (independent) had refused to resign on Sawant's request.
Only one non-BJP minister Govind Gaude survived the cull on Saturday.
Earlier, at a party meeting organised at the proposed site of former Chief Minister Manohar Parrikar's 'samadhi', Sardesai accused Sawant of trying to wipe out Parrikar's legacy and blamed him and the BJP's central leadership of betraying alliance partners, who had supported the BJP-led alliance government through thick and thin.
"When Stalin died in Russia, there was a period under (Nikita) Khrushchev called 'de-Stalination', when Stalin's legacy was undone. This is going on in Goa. Attempts are being made to wipe out Parrikar's legacy," Sardesai said at the meeting, calling the developments a "political rape of Goa".
Sardesai also claimed that Parrikar on his death-bed had requested him to continue his support to the BJP-led coalition, even after his demise. The former Deputy Chief Minister also accused BJP's central leaders of not having the "moral courage" to ask him to resign because of his promise to Parrikar, who died on March 17.
"I wanted to see which central leader had the moral authority or ethical vision to ask me to resign. No central BJP leader had the moral authority to tell me to resign. This is backstabbing," Sardesai said, adding that Parrikar had "died twice", once physically on March 17 and on July 13 when his legacy was "wiped out" by the BJP.
The Goa Forward leader also said his party would be formally withdrawing support to the incumbent government, before the monsoon session of the Goa Legislative Assembly convenes on July 15.
Sawant however insisted, that he had not mistreated anyone and that he had to act on the directions of the BJP central leadership. Explaining the reasons for inducting the 10 Congress MLAs on Wednesday, Sawant said that every party wants to have a full majority government.
"As per directions from the BJP central leadership, we have dropped four ministers and now BJP has a full majority government," Sawant said.
State BJP chief Vinay Tendulkar, who was also present at the swearing-in ceremony, said it had become a nationwide trend for leaders from other political parties to join the BJP.
"Leaders across the country, including Karnataka, are joining the BJP by leaving the Congress and other parties. This is a trend," he told reporters.
When asked about the disillusionment of the party cadre over induction of 10 Congress MLAs into the BJP on Wednesday, Tendulkar said, "The workers will have to tolerate this and all of us should go ahead together".
( With inputs from IANS )