Khanna, who had been sick since April with an undisclosed illness rumoured to be cancer, passed away at his family home in Mumbai after being discharged from hospital on Tuesday, reports said.
Indian Prime Minister Manmohan Singh led an outpouring of grief on Twitter where fans reminisced over his greatest movies and wished his wife and two daughters well.
"I convey my heartfelt condolences to the members of the bereaved family and countless fans and admirers of Shri Rajesh Khanna," said Singh's official Twitter feed.
Known as "Kaka" (uncle) to his fans, Khanna was not from an acting dynasty like many big Bollywood names. He was born in the city of Amritsar in the northwest of India in 1942 and enjoyed being on stage from his school days.
He made his film debut in "Aakhri Khat" (The Last Letter) in 1966 but his big break came with runaway hit "Aaradhna" (Worship) three years later, followed by a string of successes, with Khanna typically as the romantic lead.
His prominent hits of the 1970s included "Kati Patang" (Broken Kite), "Anand" (Happiness) and "Amar Prem" (Everlasting Love).
In total he appeared in more than 150 Hindi films.
Khanna's entry to the industry came at a time when fans were looking beyond fading stars of Bollywood legends such as Dilip Kumar, Raj Kapoor and Dev Anand.
He was soon getting letters written in blood by his female fans and his car was marked with lipstick stains wherever he went. There were even reports of his female admirers marrying his photographs.
British broadcaster Jack Pizzey in a 1973 BBC documentary "Bombay Superstar" described Khanna as having the charisma of Italian actor Rudolph Valentino and the arrogance of Napoleon.
Many Indian hearts were broken when he married young actress Dimple Kapadia in 1973. They had two daughters and later separated, but she returned to look after him during his final days of illness.
Despite his huge success, Khanna's star was later eclipsed by that of actor Amitabh Bachchan, who emerged in the early 1970s as an anti-establishment hero in roles as an angry young man.
Indian audiences began to lose their taste for Khanna-style romances and family dramas, while Bachchan's roles identified with the frustration of the country's youth, struggling with a lack of opportunities during a closed economy.
Khanna never quite regained his superstar status, although he did make a comeback in 1983 with two hits, including "Avtaar", a story of a father abandoned by his children. He released another 11 films the following year.
Later in the decade he moved into politics, contesting elections on a Congress Party ticket and becoming a Member of Parliament for New Delhi in the 1990s.
His later film roles were largely insignificant, although he shocked fans in 2008 when he did an intimate scene with then-unknown starlet Laila Khan in the film "Wafaa: A Deadly Love Story".
Laila, suspected to have had links with banned terror groups, was killed along with her family members last year.
Criticised as too bold for Indian screens, Khanna nevertheless said he was proud of the role.
His last frail onscreen appearance was in his first television commercial for electric appliance company Havells.