"The Diwan of Royal Court announces mourning and disrupts the official work of the public and private sectors for three days and flags at half-mast in the next 40 days," Xinhua news agency quoted the state TV public channel as saying on Twitter.
The Sultan, who ruled Oman for almost half a century, was unmarried and had no heir or had designated successor, according to media.
Last month he returned home after undergoing medical checks and treatment in Belgium. There were reports he was suffering from cancer, the BBC said in a report.
At the age of 29, Sultan Qaboos in 1970 deposed his father Said bin Taimur in a bloodless coup with British support and using its oil wealth, he then set Oman on a path to development.
According to the sultanate's Basic Statute, the Royal Family Council - comprising about 50 male members - should choose a new Sultan within three days of the throne falling vacant.
If the family cannot agree, members of the defence council and the chairmen of the Supreme Court, the Consultative Council and the State Council will open a sealed envelope in which Sultan Qaboos secretly recorded his choice and enthrone that person.
The sultan is the paramount decision-maker in Oman and also holds the positions of prime minister, supreme commander of the armed forces, minister of defence, minister of finance and minister of foreign affairs.
( With inputs from IANS )