The last-gasp drama occured despite the Blues taking the lead inside 18 seconds through teenager Sam Garvie, making his first start of the season in place of the injured Tom Raw.
And his strike beat Danny Brunskill's 18.5 second opener in the previous round win over Dunston Federation.
With on-loan goalkeepoer Nick Liversedge not allowed to play by employers Darlington, ex-Darlo defender Kev Burgess suspended, left-back Leon Scott cup-tied, and Steve Huggins and Tom Raw picking up knocks in midweek, Whitby were forced into making five changes from the side that defeated Bradford on Wednesday night.
Joint boss Phil Brumwell brought himself back, with summer signing from Spennymoor Craig Bishop making his first competitive appearance on the opposite side of defence, Andy Burton making just his fifth start of the season and veteran keeper Dave Campbell returning.
Whitby enjoyed some good news pre-match as the deal for midfielder Jimmy Beadle, previously banned by former club York City from appearing for Town in the FA Cup, was completed before Friday's registration deadline.
The Seasiders were fresh from five matches unbeaten and it showed as they made the perfect start.
It all began on the magic 18 second mark when teenager Garvie raced clear, rounded keeper Gareth Powell and sidefooted the ball over the line to put Whitby ahead.
In a game that occasionally threatened to boil over, Blyth and their big front pairing of defender Richard Pell and striker Phil Bell created little in a disappointing first half for the Northumberland outfit.
Managed by Whitby's must successful ever manager Harry Dunn, Spartans only threatened from set pieces towards the end of the half, which Town had dominated without truly testing the keeper.
The Blues replaced Garvie and Karl Charlton, who both picked up knocks with Lee Nogan and Thompson on the hour mark.
Dale's angled drive flew wide of Campbell's right-hand post on 67 minutes.
Then three minutes later, another ex-Blyth man in the Whitby ranks, skipper Richard Forster, threw himself at the ball to block successive Phil Bell near-post efforts.
Blyth almost stole it in the final 10 minutes.
First they levelled matters with a controversial equaliser nine minutes from time.
Pell appeared to impede Town keeper David Campbell as the pair contested a high ball, with Chris McCabe reacting quickest to scramble home from close range.
Referee Mr Bannister, who was in hot water recently for awarding the famous goal that never was at Watford, saw no infringement.
Early in the match, Blyth's good 50-plus travelling contingent cheered loudly when a shot of theirs rolled well wide, but to no avail.
Then as the Seasiders struggled to take in that blow, Spartans struck again three minutes later, a neat flick on deflected off the head of ex-Scarborough forward Ged Dalton from close range.
That would've been an injustice if it proved the winner, but in the five minutes of injury time came the Blues' dramatic leveller.
Town's man of the match Tony Hackworth was being held by the shirt as he advanced just outside the box.
But Mr Bannister played a good advantage, allowing Hackworth to play in substitute Thompson to fire past Powell, to send the home support into raptures and disappoint former boss Dunn.
Final score: Whitby Town 2 Blyth Spartans 2
Attendance: 403
Whitby Town: Campbell, Brumwell (Tinkler 86), Bishop, Lyth, Forster, Beadle, Garvie (Ormerod 60), Hackworth, Burton, Brunskill, Charlton (Nogan 60).
Whitby Co-operative WTFC Man of the Match: Tony Hackworth
Match Sponsors: Harland Brothers of Glaisdale.
Matchball Sponsors: Robert Harrison, builders and contractors of Glaisdale.

* WHITBY'S match against Blyth Spartans at Croft Park kicks off at 7.45pm tonight.
Admission prices are adults £9 (£1 extra in main stand) and concessions £5 (£1 extra in main stand).
Under 10s are free if accompanied by an adult.
Directions:
From Tyne Tunnel: take A19 signposted Morpeth.
At the second roundabout take the A189 signposted Ashington.
From A189 take the A1061 signposted Blyth.
At the first roundabout take the turn-off signposted A1061 Blyth.
Follow the road to second roundabout straight across (A1061), third roundabout straight across (A1061 Town Centre/South Beach).
At the fourth roundabout turn left onto A193 (Ind Estate).
At the fifth roundabout go straight across (A193 – Ind Estate) and at the sixth roundabout turn right into Plessey Road.
The ground is on the left.
The Whitby Town supporters' coach to the game is almost full – call chairman Graham Manser on 605921 for details.
* Don't miss Friday's Whitby Gazette for match report and reaction.