Two days ago, Swamy wrote a letter to Prime Minister Narendra Modi seeking an amendment to the Places of Worship Act, 1991 which provides for maintaining the "religious character" of holy structures as it existed on August 15, 1947.
The law also prohibits the conversion of any temple into a mosque and vice versa.
Swamy urged the Prime Minister to give directions to the Ministry of Law to make amendments to the Act, specifically section 4, which he claimed was "offensive and ultra virus of my fundamental rights" to freedom of worship.
"This Act was enacted by the Congress government headed by Narasimha Rao," Swamy said in his letter.
Swamy raised the issue again on Sunday while he was addressing the Ashok Singhal Smriti lecture series in Varanasi.
He said that after the Supreme Court verdict that favoured the construction of the Ram temple in Ayodhya, people are now anxiously waiting for the expansion of the Kashi Vishwanath temple and the Krishna Janmabhoomi.
Both these shrines are located adjacent to Muslim sites. While the Kashi Vishwanath temple is adjacent to the Gyanvapi mosque that even shares its boundary with the temple, the Krishna Janmabhoomi is next to the Eidgah in Mathura.
Swamy said: "These are issues that religion is a matter of fact and after Ayodhya, people are looking towards Kashi and Mathura. The RSS and VHP are already prepared to launch campaigns for these shrines and we should take steps at the earliest."
Senior BJP leader Murli Manohar Joshi, who was also present at the function, approved of Swamy's demand.
"Aap Swamy hai, aapko kahin bhi jaane ki azadi hai (You are Swamy and free to go anywhere)," he said.
Swamy, meanwhile, said that there was no need for a legal battle on the Kashi and Matura issue, adding that the centre can acquire the land in national interest.
( With inputs from IANS )