Marcus Rashford's sweet second-half free-kick brought United a 1-0 win in last week's first leg, putting Jose Mourinho's side within touching distance of the final in Stockholm on May 24.
But although United have gone 17 European home games without defeat, Celta have scored in all six of their Europa League away matches this season and Herrera says everyone in United red will have to be on their game.
"They have scored away in every game they've played in the competition so we have to be aware of that," the Spanish midfielder said.
"They are a very good team, very dangerous when they play away and we will need our fans at home.
"I tell them we really need them to create a good atmosphere, a difficult atmosphere for the opponents, who can push us to be in the final. It's not going to be easy."
United have never won the Europa League and are bidding to reach a first continental final since their 3-1 loss to Barcelona at Wembley in the deciding match of the 2010-11 Champions League.
But while success in Europe's second-tier competition will complete United's collection of trophies, what motivates them above all is the place in next season's Champions League that Europa League success will yield.
Mourinho has openly abandoned United's hopes of securing the top-four finish in the Premier League that would open the door to the Champions League.
He made eight changes to his starting XI for last Sunday's 2-0 defeat at Arsenal, which brought an end to United's 25-game unbeaten run in the league and left them four points below fourth-place Manchester City.
Eric Bailly, Daley Blind, Antonio Valencia, Paul Pogba and Rashford are among the players expected to return for United on Thursday.
Celta need not look far for inspiration, having overcome a 1-0 home defeat against Shakhtar Donetsk to prevail 2-1 on aggregate in the last 32.
They can also call upon cup pedigree that saw them overcome Real Madrid in the quarter-finals of this season's Copa del Rey, a year after they had accounted for Atletico Madrid at the same stage of the tournament.
"It is still close," said Celta striker John Guidetti, who was formerly on the books of United's cross-town rivals City.
"It is half-time. They are leading 1-0 and we have to go out and make the second half count.
"At the end of the day, if we go to Old Trafford and score the first goal, we are in the driving seat because if we score another one, they have to score two."
Celta's Europa League commitments have had a striking impact on their league form, largely due to coach Eduardo Berizzo's determination to rest players.
Their 3-0 defeat at Malaga last weekend was their sixth loss in seven La Liga outings and they have won only two of their last 10 games in all competitions.
Peter Bosz's dazzling young Ajax team are likely to be awaiting the victors in the final, having beat Lyon 4-1 in the home leg of their semi-final.
But Lyon have averaged over four goals per game in their Europa League home matches and coach Bruno Genesio believes his side's 3-2 victory over Nantes on Sunday showed they are capable of pulling off a famous comeback.
"There's an hour and a half to overcome our deficit," he said.
"We'll have to create chances and take them, but also remain calm and organised when we lose the ball.
"We have to remember that we scored three goals. And if we score three on Thursday, we could go through."