Alexander, who bagged a hat-trick on Boxing Day against Brighton only to break his nose two days later against Crewe, came off the bench at half-time and snapped up an injury-time equaliser when he steered home a shot by Marcus Bignot.
Northampton had led thanks to a Dean Bowditch goal after 37 minutes, but the other major talking point was the double sending-off of Poul Hubertz and Neil Harris by referee Graham Horwood ten minutes from time.
Hubertz fouled Zak Whitbread and was immediately assaulted by Harris, a former team-mate, as Hubertz only left the New Den at the end of last season.
The dismissal of Hubertz was probably of greater damage to Northampton who will miss the striker at Leeds on Saturday, a match where Daniel Jones and Brad Johnson will also be likely absentees.
Jones has come to the end of a three- month loan spell from Wolves, while Johnson was left out of this contest after Northampton received a substantial bid from a Championship club for his services.
The goal which put the home side in front was a sparkling affair, coming from a slick five-man move involving Andy Kirk, Hubertz, Danny Jackman and Jason Crowe, before Bowditch applied the finishing touch from a few yards out.
Northampton should have put the game to bed but didn't cash in on their chances and the introduction of Alexander for the injured Ahmet Birkovic gradually altered the pattern.
Goalkeeper Mark Bunn did well to keep the visitors at bay, notably by tipping an Alexander header against the bar 20 minutes from time.
Northampton could have sealed it when Bowditch sent a volley whistling wide, but the late flare-up only triggered more Millwall pressure and Alexander gleefully grabbed the equaliser.