After falling behind to James Vaughan's composed finish in the 64th minute, Lloyd Kerry levelled five minutes later, striking low with his left into the bottom right corner and in doing so claiming Town's first competitive goal as an EFL club.
Despite Joe Cracknell keeping out two of Rovers' first five kicks, the penalty tally remained level until Manny Monthe stepped up to take the hosts' tenth attempt and blazed over the bar, paving the way for Falkingam to convert Town's final kick and send his side into Sunday's draw.
Simon Weaver handed a Town debut to Tom Walker on the left of midfield, while Jack Muldoon partnered Aaron Martin up front.
In the home dugout, Mike Jackson took charge of his first Tranmere fixture and handed club debuts to Calum Macdonald, Otis Khan, Paul Lewis and former Liverpool midfielder Jay Spearing.
Town made an encouraging start and looked the sharper in the first ten minutes, with Muldoon's low cross from the right narrowly missing Martin who had looked to apply a first time finish.
Muldoon was at the heart of things again midway through the half, hooking across goal into a dangerous area, though Tranmere forward James Vaughan carried out his defensive duties well, headling clear the danger.
An interesting battle was developing in midfield between Josh Falkingham and Spearing, with Town's Captain getting the better of the two in the 34th minute, drawing a foul and the afternoon's first booking for his opponent.
The first attempt of a so far even contest came through Corey Blackett-Taylor in the 35th minute, but didn't trouble Joe Cracknell in what turned out to be the winger's last action being withdrawn due to injury moments later.
The last notable action of a first half frequently broken up by the Referee's whistle came as Muldoon threw himself at Ryan Fallowfield's cross, but couldn't direct his diving header on target.
The hosts created the first opening of the second half as Otis Khan crossed towards Morgan Ferrier at the far post, but some alert Warren Burrell defending dealt with the attack.
Town responded well though with George Thomson and Muldoon both forcing Scott Davies into saves, the latter the trickier of the two.
Just moments later Tranmere opened the scoring as former Everton frontman Vaughan showed good composure to round Cracknell and roll the ball into an empty net.
It took just five minutes for Town to respond though as Kerry claimed our first competitive goal as an EFL club. Thomson had done well to control a long diagonal ball and find Kerry in the middle, who managed to fire the ball in at the near post on his left foot.
Buoyed by the equaliser, it was Town doing all the late pushing for a winner, Will Smith going on a mazy run from the back before finding Muldoon who curled just off target.
Three minutes before the end of 90 minutes, Falkingham almost won it for his side, striking one well from all of 30 yards and forcing a good diving save from Davies and sending the tie to a penalty shoot out.
Both Tranmere and Town converted three of their opening five spot kicks, sending the contest to sudden death penalties. Smith, Fallowfield, Hall and Kerry all scored to keep Town in the tie, before Falkingham struck low under the keeper after Monthe had missed the target, sending Town into Round Two of the Cup with a 8-7 win in the shoot out.
Town: Cracknell, Fallowfield, Falkingham, Smith, Burrell, Thomson, Martin (Beck 82), Kerry, Muldoon, Hall, Walker (Stead 73)
Subs not used: Emmett, Kiernan, Kirby, Lokko
Goals: Kerry 69
Tranmere Rovers: Davies, O'Connor, Monthe, Spearing, Vaughan, Ferrier (Payne 60), Blackett-Taylor (Morris 42), Khan, Macdonald, Lewis (Banks 64), Clarke
Subs not used: Murphy, Ridehalgh, Nelson, Ellis
Goals: Vaughan 64
Referee: T Neild