Liverpool will visit either Bournemouth or Burton Albion in round four after seeing off Oldham, who stunned Brendan Rodgers's side 3-2 at Boundary Park in the fourth round last season.
Iago Aspas, a close-season signing from Celta Vigo, broke the deadlock in the 54th minute by dispatching a cross from Raheem Sterling to claim his first goal for the club.
The Spaniard then hit the post, but Liverpool did not secure victory until Oldham's James Tarkowski put through his own goal with eight minutes to play.
West Ham became the second Premier League team to be eliminated by lower-league opposition in this season's competition when they were crushed 5-0 by Championship side Forest at the City Ground.
A day after Aston Villa fell to third-tier Sheffield United, West Ham were undone by a 14-minute hat-trick from Forest forward Jamie Paterson, sandwiched by an early Djamel Abdoun penalty and a late Andy Reid strike.
West Ham manager Sam Allardyce made nine changes to his starting XI and gave debuts to three young players, but he told ITV: “What choice did I have? I don't think I had one.
“I make the decisions for the benefit of everyone at the football club. We have a huge amount of problems in terms of available players.”
Sunderland overcame third-tier Carlisle United 2-1 at the Stadium of Light to take their place in round four.
Adam Johnson put Sunderland ahead with a 34th-minute free-kick and then supplied the cross from which Sean O'Hanlon put through his own goal after Matty Robson had levelled the scores.
A 90th-minute strike from 20-year-old French midfielder El-Hadji Ba completed victory for Gus Poyet's men.
David Moyes will take charge of his first FA Cup tie as Manchester United manager later on Sunday when Swansea City visit Old Trafford. – AFP