Terry Kennedy's 83rd minute header ensured Town came away with all three points after an incredible nine-goal thriller at The Martin & Co Stadium.
The hosts lead twice in the first half through Ashley Worsfold and Bradley Wells who found the net either side of Louis Jones' own goal.Town looked to have turned the contest on its head in the first 15 minutes of the second half as a brace from Joe Leesley put the visitors two goals to the good, after Dom Knowles had made it 2-2.
The introduction of Jonathan Wafula sparked a comeback from The Holy Blues though, as Craig King and Shane Clarke's goals in quick succession made it anyone's game with just over a quarter of an hour to play.
With seven minutes of normal time remaining, Kennedy ensured his first Town goal would be one to remember as he headed George Thomson's corner past Jones.
Simon Weaver selected the same squad that put four past Chorley on Tuesday night as Jordan Thewlis lined up against his former club.
Simon Ainge's first sight of goal came after nine minutes as Leesley's cross found the feet of the striker, whose shot was deflected into the arms of Jones.
It was the hosts who took the lead though as a deflected cross fell to Worsfold a few yards out where the former Town striker was able to squeeze past James Belshaw.
Trinity's striker almost made it two four minutes later with a near exact recreation of his first finish, only this time the goal was ruled out for offside.
Worsfold's strike partner Nathan Jarman was next to trouble the Town back line as a clever low corner forced Belshaw to save at his near post.
Injury to Ainge saw Knowles take to the field, and the striker made an instant impact as his shot was spilled by Jones and cleared only cleared back onto the ‘keeper before ending up in the back of the net with Jack Emmett applying close attention.
The Holy Blues regained the lead as the contest was heading into first half stoppage time as Wells' free kick from inches outside the Town box took a deflection and nestled into the bottom left corner.
It took four minutes of second half action for Town to pull level again as George Thomson's shot was parried into the path of Knowles who showed great composure to evade his marker and find the net.
Town took the lead for the first time after 54 minutes as, following a full blooded challenge from Josh Falkingham, Knowles played in Leesley on the left who fired into the top right corner from a narrow angle.
The winger found the net for a second time six minutes later after picking up possession 35 yards out and seeing his low shot take a deflection and end up in the back of the net.
With a two goal advantage the chances came thick and fast for the visitors as Thomson played through Thewlis one-on-one with Jones, who made a good save to deny the former Trinity man.
Trinity Manager Dave Frecklington introduced Wafula in the hope of sparking a comeback, and the introduction led to a goal after just five minutes.
Wafula sprinted down the right wing and pulled back towards King in acres of space to power past Belshaw and make it 3-4 with 20 minutes to play.
It only took the hosts three minutes to level though, as a deep free kick was headed back across the face of goal where Clarke was waiting to slide home.
With both sides believing they could find a decisive goal, second half substitute Kennedy climbed highest to head home Thomson's corner and claim his first Town goal in the most dramatic of circumstances to give his side the win.
Harrogate Town: Belshaw, Fallowfield, Parker, Falkingham, Middleton (Kennedy 74), Burrell, Thomson, Emmett, Ainge (Knowles 32), Thewlis, Leesley
Subs not used: Cracknell, Day, Vann
Goals: Jones OG 33, Knowles 49, Leesley 54, 60, Kennedy 83
Booked: Emmett, Middleton
Gainsborough: Jones, Jacklin, Storey, Clarke, Evans, Stainfield, Wells (Simmons 65), Richards, Jarman (Wafula 65), Worsfold, King
Subs not used: Collins, Gatter, Walton
Goals: Worsfold 13, Wells 45+1, King 70, Clarke 73
Booked: Jarman, Clarke
Referee: B Lamb