"It's not like you've got China on your border," Trump told Modi, according to the book by two Washington Post reporters, who quoted an anonymous aide.
"After Trump bungles his India-China geography and seems to dismiss the threat China poses to India, for instance, the authors write that 'Modi's eyes bulged out in surprise,'" the newspaper wrote.
"Modi's expression gradually shifted, from shock and concern to resignation," Philip Rucker and Carol D. Leonnig claimed in their book, "A Very Stable Genius," according to the newspaper.
The Washington Post report on the book says that "some of the modest details in the book end up having larger consequences" while citing this episode with the claim that it affected India's ties with the US.
The newspaper owned by Jeff Bezos, the CEO of Amazon, said the incident showed "Trump bungles his India-China geography and seems to dismiss the threat China poses to India."
An anonymous Trump aide quoted in the book "concludes Modi probably left that meeting and said, 'This is not a serious man. I cannot count on this man as a partner.'"
The newspaper said that after the meeting, the aide told the authors, "The Ind took a step back in their diplomatic relations with the United States."
However, this claim is not backed by facts.
India's diplomatic and strategic ties have continued to grow during the Trump presidency, with problems only in the area of trade.
One factor behind the deepening of ties is the potential that Trump sees in India as a strategic counterweight to China.
Modi and the US President appear to have developed close ties, with Trump accompanying him to the "Howdy Modi!" rally in Houston in September last year.
There Modi called Trump a friend and in a twist of his own election slogan, declared "Abki baar, Trump sarkar," (this time, a Trump government).
( With inputs from IANS )