Darren Bent gave Villa the lead from the penalty spot before Herd was shown a red card for an innocuous-looking clash with Jonas Olsson inside the box.
Chris Brunt sliced the spot-kick well wide but his error was soon forgotten as Olsson headed home from a corner.
The Baggies’ dominance was rewarded when Paul Scharner fired in the winner.
The Austrian’s athletic finish ensured the Baggies ended a 13-game run without a win at the home of their West Midlands rivals.
After a poor start to the season, Roy Hodgson’s men are now unbeaten in four matches and 10th in the Premier League table, level with Aston Villa on 11 points.
Villa started the game strongly and should have taken the lead after six minutes when Bent latched on to a mis-hit Barry Bannan shot – but he fired over the bar from eight yards out.
Scotland midfielder Bannan came close with an effort of his own moments later, his volley from outside the area narrowly missing the target.
Villa full-back Alan Hutton was perhaps fortunate to stay on the field when his two-footed challenge on Shane Long went unpunished by referee Phil Dowd.
The Irishman, outstanding in the 2-0 victory over Wolves last week, limped off soon afterwards shaking his head and exchanging words with a section of Villa fans.
A bad start for the Baggies got worse two minutes later when Villa were gifted a penalty.
Hesitation from Steven Reid allowed Gabriel Agbonlahor to get to the ball ahead of Ben Foster, before tumbling over the Baggies goalkeeper.
Bent sent Foster the wrong way from the spot to put Villa ahead.
West Brom enjoyed more possession as the half progressed and were handed a penalty of their own in the 21st minute in bizarre circumstances.
Olsson and Herd tussled off the ball at a free-kick before Olsson fell to the ground. Assistant referee Darren Cann signalled for a penalty and Dowd sent Herd off to the amazement of the Villa faithful.
Their shock turned to glee a moment later when Brunt drilled his penalty several yards wide of the goal.
A barely credible half was handed a final twist when Olsson towered above the Villa defence to head home Brunt’s corner and make it 1-1.
West Brom were on the front foot from the start of the second period and, after Olsson headed over from three yards, Scharner gave his team the lead.
A corner from the left fell kindly for the Austrian, who lashed in an acrobatic left-footed shot off the underside of the bar.
Alex McLeish brought on Emile Heskey and Marc Albrighton as Villa pushed for an equaliser.
But West Brom used their extra man to dominate possession and could have scored on the break when Peter Odemwingie mis-kicked in front of goal.
A tiring Villa never looked like getting an equaliser as the Baggies saw out a memorable derby win.