Addressing a packed Adriaanse community hall, Zuma said that anti-crime committees "are needed in our blocks, streets and communities".
"We have come to Elsies River to express our outrage and say enough is enough," Zuma said, adding that he believed violence against women and children could be eradicated if communities worked together with law enforcement.
Elsies River was rocked recently by the rape and murder of three-year-old Courtney Pieters whose little body was found buried in a shallow grave near Epping more than a week after she disappeared from her Pluto Street home.
In the wake of the incident, which saw a 40-year-old lodger in the Pieters family home arrested in connection with the crime, Zuma visited the family and promised the community that he would return to look into the challenges of crime and how law enforcement authorities could improve their services.
Police Minister Fikile Mbalula, who was also at the imbizo on Tuesday, had unveiled a satellite police station shortly after Zuma's first visit to the area.
"The community has asked for a police station and the minister has responded by giving a mobile kind of police station for now," Zuma told residents.
Human settlements minister Lindiwe Sisulu, Minister of Social Development Bathabile Dlamini, as well as the deputy minister of sport, Gert Oosthuizen, also attended the imbizo.
Explaining his original visit to the community two weeks ago, Zuma said that when "I heard the news that a young child of three years was murdered, raped, it shocked me. I thought I needed to sympathise with the family and the community. I thought something is wrong with the society." Zuma said crime was a cancer that needed to be uprooted from communities.
"That's why we are back here today to discuss what we can do together to fight these terrible crimes in our community," He again extended condolences following the killing of Courtney, saying she did not deserve to have her life cut short in such a cruel manner.
"Courtney's story, as cruel as it is, is not an isolated incident," Zuma said.
"The police are doing their best but they need the support of the community to solve and stop the crimes," Zuma added.
"It is difficult to police crime in the streets. It is even more difficult to police crime committed in the household.
"More than anything else we need to unite as a nation to stop criminals."
Zuma said the ruling African National Congress, at National Executive Committee (NEC) level, had resolved that communities needed to form themselves into anti-crime committees "so that we can uproot criminality".