The Clarets had gone ahead just before the half-hour mark when Jay Rodriguez converted Ashley Westwood's peach of a pass.
That was, however, as fruitful as it got for the Clarets, who had led against their Championship opposition at the break.
The Terriers drew level with just over a quarter-of-an-hour remaining when Matt Lowton was robbed of possession on the halfway line and Sorba Thomas crossed for Josh Koroma to convert from close range.
The visitors then turned the tie on its head in the 87th minute when substitute Thomas was once again the provider as defender Matty Pearson powered a header home from his corner.
James Tarkowski almost forced extra-time in the closing stages, but the centre back failed to keep his header on target when unmarked from Westwood's set-piece.
1. Nick Pope 6
Made a meal of a few clearances in the first half, with his kicking in general leaving plenty to be desired, but was otherwise quiet on the whole. Couldn't be expected to do a great deal to stop Town's goals and made a superb save to deny Holmes before their winner.
2. Phil Bardsley 6.5
The veteran full back, making only his third start of the campaign, did well up against Ruffels and Koroma, producing a number of firm but fair challenges. Became more of a test when the game was stretched and failed to pick up the run of Town's number 10 for the equaliser.
3. Matt Lowton 5
A spate of injury and illness at Turf Moor meant the defender had to be shifted over to the left-hand-side. Hadn't really put a foot wrong until costing Burnley the tie when losing possession on the halfway line in the lead up to Koroma's equaliser.
4. James Tarkowski 5
Untested in the first half as former Rovers striker Rhodes failed to make an impression. Got across well as the covering defender on occasions after the break, but caught under Sorba's corner as Pearson converted the winner and somehow missed the target when unmarked late on.