Making a historic announcement in the Rajya Sabha that triggered bedlam, Shah said: "I am presenting the resolution to revoke Article 370 in Jammu and Kashmir except the first clause 370 (1)."
Following his speech amid opposition protests, at times drowning his voice, the House was adjourned. It later resumed business.
In a separate statement, Shah said the government has proposed to reorganize Jammu and Kashmir carving out two separate Union Territories of Ladakh and Jammu and Kashmir.
He said this had been done in view of constant threats of cross border terrorism.
"There has been a long pending demand of the people to give it the status of a Union Territory to enable them to realize their aspirations," the statement said.
"Further, keeping in view the prevailing internal security situation, fuelled by cross-border terrorism in the existing state of Jammu and Kashmir, a separate Union Territory for Jammu and Kashmir is being created," he said.
Shah's announcement came after days of escalating tension in Jammu and Kashmir, particularly the Kashmir Valley, where panicky residents have been buying up essential goods amid a security build up.
Also in the Rajya Sabha, Shah whose BJP had made scrapping Article 370 an election pledge denied that it was 370 which linked Jammu and Kashmir with India.
"Opposition leaders are saying that Article 370 brought Jammu and Kashmir to India. But the fact is that the Maharaja Hari Singh signed on October 27, 1947 (the Instrument of Accession) while Article 370 came into existence in 1949.
"So this is wrong to say that Article 370 brought Jammu and Kashmir with India," the Minister told the Rajya Sabha in response to Congress leader and former Chief Minister Ghulam Nabi Azad who condemned the decision to axe Article 370.
Leader of Opposition Azad said: "By revoking Article 370 that accords special status to Jammu and Kashmir, the BJP government has murdered the Constitution of India."
He underlined that it was the historic Article 370 which linked Jammu and Kashmir to India.
Shah said Article 370 was used as "vote bank politics" and that earlier governments lacked the political will to revoke it.
"But the Modi government has the political will and we are not bothered about vote bank politics."
He asked the opposition members to have a debate on the resolution and said there should not be a delay "even one second" in revoking Article 370.
In Parliament, two members of the Peoples Democratic Party (PDP), Nazeer Ahmed Laway and Mir Mohammad Fayaz, were told to leave the House after they tore up copies of the Constitution.
But besides the treasury benches, the BJP won the support of the Bahujan Samaj Party (BSP), Biju Janata Dal (BJD), Telugu Desam Party (TDP) and YSR Congress supported the government move.
On Sunday night, the Jammu and Kashmir government imposed restrictions under Section 144 of the CrPC in Srinagar district and placed the top leadership of the state, including former Chief Minister Omar Abdullah and Mehbooba Mufti, under house arrest.
Internet services were shut down in several parts of the Valley and educational institutions were ordered closed. Earlier, tourists were asked to leave Jammu and Kashmir while the Amarnath Yatra was curtailed.
National Conference leader Omar Abdullah called the decision "a betrayal of people's trust" and amounted to "aggression".
PDP leader Mehbooba Mufti dubbed the decision to scrap Article 370 as "the darkest day in Indian democracy".
"Decision of J&K leadership to reject 2 nation theory in 1947 & align with India has backfired. Unilateral decision of GOI to scrap Article 370 is illegal & unconstitutional which will make India an occupational force in J&K," she tweeted.
"It will have catastrophic consequences for the subcontinent."
( With inputs from IANS )