Curious onlookers peer into the touilet where the two girls were found. Two children aged 2 and 3 were found dead in a toilet in Diepsloot extension 1 this morning, a massive man hunt is underway. Picture: Antoine de Ras, 15/10/13

* This story has been updated.

Johannesburg - Police are on the hunt for a Mozambican man believed to be behind the child killings in Diepsloot.

 "I can safely say that they (suspects in custody) are of foreign origin, and the main perpetrator that we are looking at would be a Mozambican citizen," said Maj-Gen Norman Taioa, head of detectives in Gauteng.

Taioa, Gauteng acting provincial commissioner Lt-Gen Lesetja Mothiba, and cluster commander of the Honeydew area Maj-Gen Oswald Reddy, were addressing the media in Johannesburg on Wednesday.

Mothiba said police would soon release an identity kit of the man.

He said police were also investigating a link between Tuesday's killings and the murder of a five-year-old girl in September.

"The team of investigators are looking at the possible link between the reported kidnapping and murder case to that of a five-year-old girl who was found dead in September in the same area."

The man who was questioned in connection with the crime was later released as he could not be linked to the killing and rape.

The bodies of two-year-old Yonelisa Mali and her cousin Zandile Mali, three, were found in a public toilet cubicle in Diepsloot at 5am on Tuesday by a resident.

They were reported missing at the weekend.

Residents went on the rampage after the bodies were discovered.

Police said a team of detectives had been appointed to investigate the murders.

Four people had been taken in for questioning. A fifth person was being sought.

Taioa told reporters that the men had gone to the police station to inquire about damage to their shack which was vandalised by residents when they protested.

"They were then taken in for questioning and it is based on that question that we are still having them for further investigations."

Reddy said all the residents of the yard where the men lived were taken in by police for questioning and to establish if they were involved in the crime.

"We questioned (them) to see any linkage between the scene of (the) crime and those individuals through DNA testing, which has been done."

He said the men knew the whereabouts of the main suspect and they were assisting police with their investigations.

Mothiba said three of the men had South African identity documents but their authenticity would be examined to see if they had been fraudulently obtained.

"After 48 hours, a decision will be taken whether we are going to charge or release them but at this present moment we have enough grounds to detain them."

He said investigations would reveal if there was child neglect involved.

"The police are also calling on parents, especially mothers, to fulfil their parental responsibility in caring and looking after their children.

"If in the investigation it is found that there was a lack of parental supervision, we will charge the parents for child neglect," Mothiba said.