The board decided to act following Saturday's 1-0 FA Cup first-round defeat by Blue Square Bet Premier promotion hopefuls Luton at Kenilworth Road.
Johnson, 56, took over at Sixfields in March 2011 when he replaced Ian Sampson, but has overseen six defeats in their last seven games in all competitions and leaves with the club sitting 20th in the table.
Chairman David Cardoza said: "Gary has been a successful manager in the past and I am sure he will be again in the future.
"Unfortunately, things just haven't worked out for Gary here and there comes a point when you have to accept that fact, take stock and make a decision and following a discussion between Gary and I over the weekend, it was decided that we part company.
"We wish Gary all the very best for the future. We part on good terms and our search for a new manager now begins."
Johnson, who enjoyed some success at Yeovil and Bristol City, added: "After discussions with the chairman, it was agreed that I part company with the club.
"I would like to thank the chairman for his support during my time here.
"I am sorry we did not reach the expectations we had set ourselves but I wish the club, it's players and supporters all the very best for the future."
David Lee, assisted by Andy Holt, will take caretaker charge.
Source: PA
Source: PA