Traditionally, government raises cheaper funds from international markets by offloading sovereign bonds.
The move will free up additional liquidity in the domestic market. However, it is expected to impact the Indian rupee and G-sec Bond yields.
"India's sovereign external debt to GDP is among the lowest globally at less than 5 per cent," Sitharaman said in her maiden Budget speech in Parliament on Friday.
"The government would start raising a part of its gross borrowing programme in external markets in external currencies. This will also have beneficial impact on demand situation for the government securities in domestic market."
Finance Secretary Subhash Chandra Garg at the post-Budget media conference said: "We will be starting the sovereign bond preparations soon. We have not decided the exact amount, but we will be starting the process soon, certainly in this financial year."
A sovereign bond is a debt security issued by a national government. Sovereign bonds can be denominated in a foreign currency or the government's domestic currency.
It will be a maiden such bond issuance. In 2013, the government had considered the idea, but never implemented it. The country that time faced wide fiscal and current account deficits.
Instead, the Reserve Bank of India announced a scheme to incentivise foreign currency non-resident (FCNR) deposits, which brought in nearly $34 billion.
Consequently, most of India's debt is rupee-denominated.
The share of government debt in total external debt is minuscule. India's sovereign external debt stood at $103.8 billion at end-March 2019, showed the latest data released by the Reserve Bank of India in June.
The share of government debt in total external debt stood at 19 per cent at end-March 2019 from global financial agencies, the data showed.
Besides, the move is prudent in the face of limited options to raise funds as a slowing economy curtails tax revenue, while the borrowing target of a record Rs 7.1 trillion ($104 billion) this fiscal year remains a tough task.
On Thursday chief economic advsior Krishnamurthy Subramanian had said government should look at raising capital from low interest markets like Japan and European countries to lower the fiscal deficit.
India Inc welcomed the announcement. Industry body CII Director General Chandrajit Banerjee said: "Government's intent to raise a greater part of its borrowing requirements internationally, will have a positive effect on government yields with a benign impact on interest rates."
"It will also reduce the crowding out the effect of government borrowing, making more capital available for private investments."
Lakshmi Vilas Bank Head of Treasury R.K. Gurumurthy said: "Fiscal deficit and gross borrowings are in line with pre-Budget expectations. Additionally, some part of borrowings within the gross borrowings is estimated to be raised overseas. So the pressure on domestic liquidity is that much less."
The Indian rupee can gain a bit on the back of this development. However, most of today's reactions could reverse as timing is the key, he added.
"Bond yields were expected to touch 6.50 per cent and today's low was close to that. Bias remains for softer yields this quarter," Gurumurthy said.
( With inputs from IANS )