Mathur was first administered the oath by Chief Justice of Jammu and Kashmir High Court, Geeta Mittal, at Leh.
This was followed by the oathtaking ceremony of Murmu at Raj Bhavan in Srinagar.
The ceremony was kept low key despite being a historic occasion and the media was not invited.
Senior officials from the civil administration and security, including advisors to former Governor Satya Pal Malik, GOC, 15 Corps, Lieutenant General K.J.S. Dhillon and BJP MP from Jammu Jugal Kishore attended the ceremony.
Malik has been made Governor of Goa for the rest of his tenure.
Murmu's wife and niece were also present at the ceremony.
Born in Odisha in 1959, 1985-Gujarat cadre IAS officer Murmu has served as Principal Secretary to Prime Minister Narendra Modi when he was Chief Minister of Gujarat.
After the scrapping of Article 370 on August 5, Home Minister Amit Shah told Parliament that the statehood of Jammu and Kashmir would be restored when the situation is conducive. Ladakh, however, will remain a UT.
After the revocation of Article 370, a lockdown and communications blockade was imposed in Kashmir that was eased later.
There has been a general shutdown in Kashmir against the scrapping of Article 370 which has taken a toll on business and daily life. Public transport is also off the roads since August 5. Mobile and high speed internet continues to remain suspended. Post-paid mobile voice calls have been restored but the same is yet to be restored for pre-paid service.
Key political leaders of Kashmir like former Chief Ministers Omar Abdullah, Farooq Abdullah and Mehbooba Mufti continue to remain under preventive detention since the scrapping of Article 370.
( With inputs from IANS )