Substitute Sylvester Jasper struck in the 88th minute to send Town to a second defeat of the season at Colchester's JobServe Community Stadium.
In a contest that didn't come to life until late in the second half, it was Jasper who was able to take the chance for his side, scoring the only goal of the game that sent Town home empty handed.
Simon Weaver named the same starting XI that recorded a record win against Scunthorpe last weekend, though there was a place on the bench for Aaron Martin who returned from injury.
The hosts enjoyed the first threatening bit of play when Freddie Sears won a free kick in a good position that Tom Eastman couldn't head goalwards when delivered into the Town area.
Town had the ball in the net in the tenth minute when George Thomson tapped into an empty net, but Luke Armstrong had been flagged offside in the build up.
Weaver's side's best chance of the half came on the half hour mark when the impressive Will Smith got his head to Thomson's corner, but guided the ball into the arms of Shamal George.
Town's centre halves were having the better of things against Frank Nouble and Sears, though Connor Hall did pick up a first half caution for a trip on the latter.
With neither side able to make a dent in the scoreline in the opening 15 minutes of the second half both sides opted for changes with Lloyd Kerry and Jasper entering the action.
Town's number 17 almost made an instant impact after Jack Diamond had dribbled round his marker and found his teammate, though's Kerry's effort rattled the cross bar and fell the wrong side of the goalline, the closest either side had come to scoring so far.
Colchester responded with their best chance of the game as the ball was worked to Frank Nouble, but a heroic Ryan Fallowfield block just about stopped the U's forward hitting the target.
Town were applying pressure to the Colchester goal, but still couldn't find a way through, and that almost proved costly in the 86th minute as Jasper struck the cross bar from a free kick.
Colchester had looked most dangerous on the counter attack and managed to make it count in the 88th minute, Brendan Wiredu carrying the ball forward and feeding Jasper who slid the ball under Oxley.
Aaron Martin and Danilo Orso were intorduced in an attempt to force an equaliser, but Colchester held firm to claim the win.
Harrogate Town: Oxley, Fallowfield, Falkingham (Martin 90), Smith, Burrell, Thomson (Kerry 62), Pattison, Muldoon, Hall, Diamond, Armstrong (Orsi 90)
Subs not used: Cracknell, Sheron, Power
Booked: Hall, Thomson
Colchester: George, Chambers, Wiredu, Hannant, Skuse, Nouble (Tovide 83), Sears, Chilvers (Jasper 63), Eastman, Tchamadeu (Smith 90), Daniels
Subs not used: Turner, Dobra, Kennedy, Coxe
Goals: Jasper 88
Referee: T Reeves
Attendance:2,439 (93 away)