The 52-year-old is believed to have suffered a heart attack and died in the early hours of Tuesday morning.
London Mayor Boris Johnson, with whom he clashed over plans for the Tube, said: "I am shocked. Bob Crow was a fighter and a man of character."
Labour leader Ed Miliband said he was "a major figure in the Labour movement and was loved and deeply respected by his members".
Mr Crow was elected general secretary of the RMT in 2002 following the death of former leader Jimmy Knapp.
The BBC's political editor Nick Robinson said Mr Crow "was, some argue, the most successful union leader in terms of securing jobs and pay for his members".
He is understood to have been due at a TUC executive awayday in Surrey on Monday, but was unable to attend because he was feeling unwell.
Mr Miliband offered his condolences: "I didn't always agree with him politically but I always respected his tireless commitment to fighting for the men and women in his union.
"He did what he was elected to do, was not afraid of controversy and was always out supporting his members across the country.
Bob Crow was admired by his members and feared by employers”
TSSA union leader
"He was a passionate defender of and campaigner for safe, affordable public transport and was a lifelong anti-fascist activist."
Mr Johnson said in a statement: "Whatever our political differences, and there were many, this is tragic news."
"Bob fought tirelessly for his beliefs and for his members.
"There can be absolutely no doubt that he played a big part in the success of the Tube, and he shared my goal to make transport in London an even greater success.
"It's a sad day."
Former mayor Ken Livingstone told Sky News: "He fought really hard for his members. The only working-class people who still have well-paid jobs in London are his members."
Bob Crow was a vocal campaigner against the war in Iraq
Born in 1961 in east London, Mr Crow got his first job on the underground at the age of 16, fixing rails and cutting down trees by the track.
He became a local representative for the then National Union of Railwaymen at the age of 20.
Boris Johnson: "This was a guy who really fought for his members"
Last month, Mr Crow joined his members on the picket line during a Tube strike, called in protest at the mayor's plan to close ticket offices.
Manuel Cortes, leader of the TSSA rail union, which also took part in the strike, said: "Bob Crow was admired by his members and feared by employers, which is exactly how he liked it.
"It was a privilege to campaign and fight alongside him because he never gave an inch."
In an interview broadcast on BBC Radio 4 on the day before he died, Mr Crow described himself as "talkative", but said he didn't like to be "gobby".
"At the end of the day, to be a general secretary of a union you've got to be larger than life," he said.
"[You cant] walk around with a grey suit on and eat a cheese sandwich every lunch time. You want someone who's got a bit of spark about them."
TUC general secretary Frances O'Grady said he was "an outstanding trade unionist, who tirelessly fought for his members, his industry and the wider trade union movement".
Paul Kenny, GMB general secretary, added: "Even people who didn't like what he did agreed he did it very well."
Kevin Maguire, associate editor at the Daily Mirror, said on Twitter: "Bob Crow was a big burly bloke with a great brain, wit, heart of gold and surprisingly soft handshake. Missing him already."
A longstanding Millwall supporter, Mr Crow was not a member of any political party when he died, although earlier in his life he did belong to the Communist Party of Great Britain.
He was often equally critical of Labour - especially New Labour - as he was of the Conservatives.
Nevertheless, many Labour MPs gave glowing tributes to him.
John McDonnell, convenor of the RMT parliamentary group, wrote on Twitter: "In Bob Crow we have lost one of the finest trade union leaders and socialists our movement has known. I am devastated by this tragic news."
His colleague Jeremy Corbyn said Mr Crow was also "a very dedicated opponent of racism and fascism whenever it reared its head", as well as taking part in anti-war campaigning.
Lindsey German, convenor of the Stop The War Coalition, said: "He was on all the major demonstrations against the war in Iraq, and was consistently against our government's foreign policies.
"I remember him on the front line of our demo in September 2002, when he helped control the massive crowd."
Source BBC news.