The entire area looked like a fortress with police and paramilitary forces deployed in large numbers. Police said three people had been arrested for rioting.
Trouble erupted on Sunday night over the parking of a scooty and quickly took a communal turn.
Delhi Police and residents appealed for calm on Tuesday even as small groups of protesters continued to raise derogatory slogans.
Hauz Qazi is a wholesale hub for hardware products.
Union Minister and area MP Harsh Vardhan who visited the area tweeted: "I visited the old Durga Temple in Lal Kuan area. The vandalising of idols inside the temple was disheartening."
He said he had been assured that none of the culprits would be spared.
Police officials were scanning CCTV footage to identify others who were involved in the violence.
There was minor stone pelting earlier on Tuesday but the trouble was quickly put down by the authorities.
Congress spokesperson Randeep Singh Surjewala urged people not to be taken in by rumours and said Home Minister Amit Shah and the Modi government were responsible for the violence as Delhi Police reported to the Centre.
"Despite police presence, the situation is not normal. People get agitated at the drop of a hat. The slogans raised by both sides are demeaning," a resident told .
( With inputs from IANS )