Some of the families had shelled out more than R200 000 for the trip.
And some families only found out at the airport on Tuesday that their visas had not been arranged.
Adam Essop said he had paid about R72 000 for the trip to the holy city. Some of the family members who were supposed to travel included his daughter, son-in-law and granddaughter.
“It is very disappointing This is unacceptable. What kind of travel agent is this?” wondered the irate Essop.
What angered him most was that he had made the travel arrangements months in advance.
He found it heartbreaking to see other people leaving on their flights while their group, which used Malek Travels, was left behind trying to work out what had caused the disaster.
Essop, who has made the pilgrimage on 19 previous occasions, said this was the first time he had encountered such a situation.
Another pilgrim who had expected to be on his way to Mecca was Gulzaar Essop.
The Pietermaritzburg resident said he had organised the trip for 13 family members at a cost of than R200 000.
A fuming Gulzaar said the flight was supposed to go to Dubai and then Saudi Arabia.
“They have been promising us for the past two weeks that everything would be all right,” Gulzaar said.
He said people from around the province had arrived at King Shaka International Airport, only to be told they were not able to travel because there were no visas.
They already had their flight tickets in hand.
He said Malek Travels offices were filled with angry people who wanted to know how this could have happened.
“This is terrible,” Gulzaar said.
He said: “They (his family) are looking to me for answers and I have nothing to give.”
He had been calling the travel agency for the past two weeks asking about the progress of their visas, and was constantly told they would receive them soon.
Gulzaar said he had gone to the airport in the belief that they would finally receive their visas there, but it was not to be.
It was not only the botched visa issue that had made him angry but that he had taken leave from work to make the trip. He was embarrassed to have told people he was travelling, only for him not to go in the end.
Abdulaziz Wahab, of Malek Travels, said: “There was a delay with the visas. They (the Saudi authorities) normally do not delay.” He said the visas were obtained from the Saudi Arabian Embassy in Pretoria and normally took about three to four days to be obtained.
Wahab said he did not understand what had caused the delay. If they were unable to remedy the situation, they would reimburse people for their costs, he said.