Rovers had several first-team players unavailable due to a winter sickness bug but they still put in a spirited performance on the road at mid-table Southern League Premier Division outfit Hitchin.
The opening half was a cagey affair, mostly played in the middle of the park.
It was the hosts that broke the deadlock just before the quarter-hour, after a missed interception allowed Hitchin to break down the left-hand channel. The ball was played across goal where Brett Donnelly was waiting to tap home from eight yards.
Thereafter, the half continued in its industrious midfield vein. Hitchin pressed Rovers on the ball and forced turnovers in possession while Rovers’ backline stood resolute to the efforts of the hosts.
Setchell feels Bedford Town deserved their opening point at Stratford
Bedford's women cricketers make it three wins in a row
Clubs all set for big kick-off as Bedford Town, Ampthill Town and Real Bedford begin campaigns
Bedford high jumper Gracie takes silver in Belfast
Al Boum Photo joins an illustrious list of back-to-back Gold Cup winners
It wasn’t until the 51st minute that Rovers registered their first proper effort of the night when George Boland warmed the gloves of Michael Johnson in the Hitchin goal from the edge of the box.
Kempston perhaps should have levelled just before the hour when Luke Heneghan played a delicious cross field through ball to Jake Newman.
However, the side’s leading scorer saw his effort smothered by Johnson as he rushed from his line to deny the one-on- one situation.
Shortly after, Newman hit the side netting after rounding Johnson from an extremely tight angle.
But as Rovers pressed forward for the equaliser, they were left vulnerable to the counter attack and Hitchin broke with pace and purpose.
From a late corner following one of these breaks, substitute Patrick Tshikala scuffed a volley across the Rovers’ box to the far post where fellow replacement Connor Vincent was there to stab the ball over the line from close range to make it 2-0 and settle the tie.
It meant that – strangely – Rovers bowed out of the competition for the second time this season, after they were reinstated following Royston’s expulsion due to fielding an ineligible player.
But Rovers can still hold their heads high as the depleted side more than matched a strong Hitchin team from a higher division.
This Saturday, it’s back to league action as Rovers play host to Marlow at Hillgrounds with a 3pm kick-off.
Rovers: Conway, Goodman, Mercer, Boland, Simpson, Collard, Tshikuna, Heneghan, Newman, Ivy, Fuller.
Substitutes: Acquaye, Stoyles
MOM: Seb Simpson.