The big Fermanagh man struck in the 93rd minute with a near post tap-in after Niall McGinn's effort from the edge of the area had been pushed out by 'keeper Gábor Király.
Northern Ireland had been trailing to a 74th minute Richárd Guzmics goal.
The Wisła Kraków defender pounced on the loose ball following a Michael McGovern spill from Balázs Dzsudzsák's free-kick.
And it got worse for Northern Ireland seven minutes after that when they were reduced to ten men following Chris Baird's dismissal for picking up two cautions in very quick succession.
Hungary hadn't tasted defeat in competitive action since the meeting in Budapest between the sides exactly a year ago and this form demonstrated itself as early as the fourth minute when Dzsudzsák fired just wide of the left post.
With the stakes so high it was perhaps no surprise that chances at either end proved to be something of a rarity with Hungarian 'keeper Király only called into action to punch away a Chris Brunt free-kick while McGovern needed to be alert to deal with a Krisztián Németh effort midway through the first half.
The game opened up considerably after the break and while realistic scoring efforts were initially still at a premium Jonny Evans will feel he could have done better when presented with a header from an Oliver Norwood corner on 54 minutes.
Norwood then forced Király into a save just before the hour with a 30-yard drive.
Hungary began to create openings of their own with Zoltán Gera firing wide before Zsolt Kalmár forced McGovern into tipping his free-kick over a few minutes later.
But the game will forever be remembered for that dramatic last 20 minutes or so when the vociferous home crowd were subjected to a myriad of emotions in a gripping finale.
Northern Ireland remain top of Group F following Romania's scoreless draw at home to Greece while Finland needed just one goal to defeat the Faroe Islands in Helsinki.
Northern Ireland: McGovern, McLaughlin, McAuley, J. Evans, Baird, Davis, Lafferty, Brunt, C. Evans, Dallas, Norwood subs Carroll (not used), Mannus (not used), Ferguson (replaced Dallas 84), McGinn (replaced C. Evans 56), Grigg (not used), McCourt (not used), McNair (not used), Hughes (not used), McKay (not used), Cathcart (not used), Magennis (replaced Norwood 75), Lavery (not used)
Hungary: Király, Leandro, Kádár, Fiola, Elek, Dzsudzsák, Szalai, Gera, Németh, Kalmár, Guzmics subs Dibusz (not used), Bogdán (not used), Lang (not used), Bódi (not used), Lovrencsics (not used), Bosnjak (not used), Nagy (replaced Elek 23), Nikolić (not used), Stieber (not used), Priškin (replaced Szalai 68), Vanczák (replaced Németh 89), Márkvárt (not used)
Referee: Cüneyt Çakιr (Turkey)
Kyle Lafferty's goal from behind the net
Gareth McAuley and Jonny Evans with Hungary's Tamas Kadar
Steven Davis and Hungary Adam Bodih in action