The Blues made two changes to the side which lost 2-0 at Frickley Athletic in midweek, with Ashley Lyth returning to the defence and fellow full-back Alex Janes making his first appearance of the season.
And the visitors made an encouraging start with Danny Brunskill and Tony Hackworth influential early on.
But it was Matlock who enjoyed the first clear-cut chance on 15 minutes. Lee Pallender's blind backpass enabled Jamie Jackson to nip in and steal the ball, but after being forced wide, the ex-Chestefield youngster saw his low drive smothered by Ben Escritt.
Then, on 35 minutes, Pallender's touch fell short, this time from a stooping header, but the towering figure of Dene Cropper lashed the loose ball wide of Escritt's near post.
Moments later, Cropper nodded wide from a right-wing cross and just before half-time, Escritt reacted well again to cling on at his near post, to another lively effort.
And despite a sedate opening to the opening 10 minutes, the second half continued in the same vein, with Jackson seeing his far post headed goal chalked off for offside on 55 minutes.
Nine minutes later, some poor play from the Matlock midfield saw veteran Mark Tinkler gifted possession 25 yards from goal.
The former Leeds United and Hartlepool man lobbed keeper Andy Richmond but the ball drifted narrowly high and wide.
On 67 minutes, Janes cut in from the left and unleashed a piledriver from the edge of the penalty area that brought a fine tip over from Richmond.
The resulting corner was laid off to Steve Huggins who battered an even better goalbound effort from distance, and the Gladiators' keeper was forced into another superb last gasp save.
Whitby kept up the onslaught and even when Matlock attacked, the Blues caught the hosts on the break on 71 minutes.
Karl Charlton forced his way through and brought a partial block from Richmond with an angled drive that found its way through to Tony Hackworth, who fired wide from a tough angle.
The breakthrough finally came within seconds, but it was at the other end when Lyth misjudged a long ball, allowing it to bounce over him and release Ross Hannah who patiently waited for the perfect time to despatch the ball under Escritt on the half volley for 1-0.
Pallender completed an unwanted hat-trick of errors moments later when an awful touch put Cropper clear on goal, but the former Worksop player inexplicably miscontrolled with the goal completely at his mercy.
Tom Raw replaced Pallender, and with the Seasiders frantically scrapping for an equaliser, Hackworth's appeals for a penalty fell on deaf ears as he twice knocked the ball against the hand of full-back Liam King on the right side of the Matlock box.
Substitute Raw then missed a great chance to level the scores with 10 minutes remaining.
After a long scramble, the 23-year-old rounded Richmond but saw his shot charged down on the goal line and hacked clear.
This proved a pivotal point in the match as play quickly switched to the other end and Hannah raced clear, only to be stopped in his tracks by Escritt's full-length dive.
Referee Mr Ledger adjudged the ball was not taken by the keeper, and the ex-Sheffield United and Wednesday junior was given the chance to put the result beyond doubt from the penalty spot.
An emphatic left footed drive from 12 yards inside Escritt's left-hand-post did that seconds later.
Whitby Town: Escritt, Lyth, Janes, Pallender (Raw 76), Forster, Tinkler (Thompson 82), Huggins, Burton (Nogan 82), Hackworth, Brunskill, Charlton. Subs not used: Brumwell, Ledgeway.
Result: Matlock Town 2 Whitby Town 0