The five-time champions made 388 and then saw the Black Caps come agonisingly close to pulling off the second highest chase in one-day internationals.
They finished on 383-9 having started the final over needing 19 to win and then seven off the last two deliveries before key batsman Jimmy Neesham was run out.
"That was awesome," said Cummins. "Sometimes I have to remember I'm out in the field, not a spectator. A fantastic game, they kept coming at us."
Australia's win had been set-up by a 175-run opening partnership between Travis Head, who top scored with 109 on his return to the team after breaking his hand, and fellow opener David Warner (81).
"I love the openers, especially Travis who's been away for five weeks," added Cummins.
"It was a good wicket. In patches we bowled really well, sometimes we gave too much width away. Some really good partnerships, it was tough to get into them."
Both Australia and New Zealand, with four wins and two defeats apiece, are well-placed to make the semi-finals.
"It was a fantastic game of cricket. There were ebbs and flows throughout the 100 overs. Obviously to get so close hurts," said New Zealand captain Tom Latham.
"You need to play the perfect game when you are chasing near 400."
New Zealand battled hard with Rachin Ravindra making 116, Daryl Mitchell adding 54 and Neesham supplying late fireworks with a 33-ball fifty.
With a total of 771 runs, it was the highest scoring game at a World Cup, beating the 754 scored in South Africa's win against Sri Lanka earlier in this tournament.
"It was a fantastic knock from Rachin, one of the better ones you'll see in a chase. It was pretty special to get close. I'm proud of the guys," added Latham.