Chasing 246 to win from a rain-reduced 43 overs after Edwards made a crucial 78 not out, South Africa were bowled out for 207.
"Proud of the group. Putting in that performance and being able to play my part is nice," said Edwards after their first win of this tournament followed on from two losses.
"When we came into the tournament we wanted to have a chance at playing the semi-finals and if you want to do that you've got to beat teams.
"Obviously South Africa are probably one of the favourites the way they are playing, so it is a big win for us."
🏏 NETHERLANDS DEFEAT THE PROTEAS— Proteas Men (@ProteasMenCSA) October 17, 2023
🇿🇦 David Miller (43) fought hard but it was not to be as the Netherlands earned a victory over the Proteas 🏏
Not the result we had hoped for but we will comeback stronger on Saturday 💪 🇿🇦#CWC23 #SAvNED #BePartOfIt pic.twitter.com/iJijVaeRvr
The victory was only the third for the Dutch at a World Cup after seeing off Namibia in 2003 and Scotland in 2007.
Tuesday's win was their first over a Test-playing nation at the 50-over showpiece.
However, the Dutch had stunned South Africa at the Twenty20 World Cup in Adelaide last November.
Edwards saluted the role of all-rounder Roelof van der Merwe who played 13 ODIs for South Africa before moving to the Netherlands.
Van der Merwe, 38, hit a quickfire 29 off 19 balls with three fours and a six as he and Edwards put on 64 for the eighth wicket after the Dutch were struggling at 112-6.
The veteran also had figures of 2-34, getting rid of Bavuma and the dangerous Rassie van der Dussen.
"He is good fun to bat with. He runs hard and hits it in quite strange areas, which is good to watch and I'm sure it is from the outside as well," said Edwards who also praised tail-ender Aryan Dutt's 23 from nine balls which featured three sixes.
"I just got him on strike. He was just clearing the ropes. It is some of the best hitting I have seen in a long time, especially from a number 10. He has got the talent and it was awesome to see," said the captain.
South Africa went into Tuesday's game on the back of impressive wins over Sri Lanka and five-time champions Australia.
But they lost Quinton de Kock, Aiden Markram and Van der Dussen, who had all hit centuries against Sri Lanka, with the score at 44-4 before they were reduced to trying to bat out their overs to limit the damage to the net run-rate.
"Credit to them, they exploited some weaknesses in our game," said Bavuma whose side conceded 32 extras when they bowled, 21 of them wides.
"We were clinical against Australia but the challenge is always to come back and replicate it. The extras is something we can control and giving away 30 extras is an extra five overs and it will always cost. It is something for us to talk about. In the end it cost us quite a bit."
He added: "We'll have to let the emotion seep in. There is no point trying to forget what happened. It will hurt and it should hurt. Tomorrow we'll get back on the journey, our campaign is by no means over."