The incident happened at Parliament's International Relations section which is located on the second floor.
According to insiders, the man had been agitated because his contract for employment had not been renewed.
An official at Parliament has also highlighted the security breach as staffers questioned how a gun could be brought into the precinct.
The man, from the Eastern Cape, is believed to have entered the building through the underground parking.
Another staffer said the man's manager, with whom he had a grievance, was not present at the office.
"If he had been there, there could possibly have been a bloodbath," said the worker.
Ambulances and police descended on the parliamentary precinct, and first blocked off the parliamentary entrance to 90 Plein Street. This was later re-opened.
Currently, paramedics and police are still on the second floor of the building conducting forensic tests.
Shooting at Parliament. Apparently a man shot himself on the second floor. pic.twitter.com/avRANJyFvn
Parliament issued a statement saying its Presiding Officers were "shocked and saddened at the passing away this morning of a parliamentary official who allegedly took his own life at Parliament".
"Parliament is in the process of informing the deceased official’s family and the Presiding Officers will visit them.
"The incident has, obviously, caused trauma and distress among parliament's members of staff and the institution as a whole. Necessary assistance is being arranged to help them to come to terms with the tragedy."
For a suicide-related emergency, contact SADAG on 0800 567 567 or their 24hr Helpline 0800 12 13 14.