Remembering the leader, Modi, who is in New York for the UN General Assembly, shared a video and said that if anyone spoke about Upadhyaya, "the first thing which comes to mind is simplicity".
In the 1.45-minute video, Modi said: "The poor were the focus point of Pandit Ji's thoughts. He believed that the politics, economic and social policies related to the ground can change the faith of the nation. He wished for a country where there is no discrimination or exploitation. Pandit Ji taught us to keep moving in life and this is s big lesson.
"Paying homage to one of the greatest icons of India, Pandit Deendayal Upadhyaya. His life's message of serving the most downtrodden with compassion resonates far and wide," Modi tweeted.
Shah said Upadhyaya's ideology was not to come to power but to rebuild the nation and restore India to its glory.
"The seeds of ideas and principles sown by him gave an alternative ideology to the country," the Union Home Minister said in a tweet in Hindi.
Shah said as a Bharatiya Janata Party worker, they were proud that in the last five years, the Modi government has followed Upadhyay's philosophy for the welfare of the poor.
BJP Working President J.P. Nadda cleaned and garlanded a statue of Upadhyaya opposite of the party headquarters here.
Nadda was accompanied by Lok Sabha MPs Manoj Tiwari and Harsh Vardhan along with other party leaders.
Union Defence Minister Rajnath Singh also paid his tributes and said: "I salute and remember Upadhyaya, a great human, thinker, philosopher and preacher.
"We are constantly moving forward on the path he paved for public welfare and poor welfare."
Upadhyaya was born on this day in 1916 in Mathura.
( With inputs from IANS )