The 57-year-old South African – who was the national side's wicketkeeper for seven years when they were welcomed back into the international arena in the 1990's – said that added exposure in the Olympics would help to globalise the game.
"I think a majority of the (ICC) members and certainly myself think the time is right," Richardson told a small group of journalists after speaking at the SportsPro Live conference in London.
"I think we have come to the conclusion the overall benefit to globalise it and grow the game will outweigh any negatives.
"We need to make a decision I would guess by July this year because I believe the IOC will in September be considering new sports for 2024."
Appropriately one of the two remaining candidates to host the 2024 global sporting extravaganza is Paris where cricket was last played at a Games in 1900 – Great Britain beating the only other entrant France by 158 runs.
"Neither Los Angeles and Paris would be disasters and and each might hold opportunities for us, especially the United States option," said Richardson.
"In Europe, too, it would be good although we would have to spend a bit more on cricket pitches in France than the US but it's not impossible."
Richardson, who has been in his post since June 2012, admitted the scheduling, especially if held in Paris, would be the biggest challenge.
"From an ICC perspective the most challenging part is the scheduling as the Olympic Games is normally held in the Northern Hemisphere summer," said Richardson.
"It could be a problem for England, who for instance wouldn't want to send their best players during an Ashes series and they have raised that concern in the past."
Olympics demands top players
That could be problematic as one of the IOC's stipulations to sports wishing to gain entry is that the best players turn up – golf's return to the Games last year was marred with several of the top stars withdrawing.
Richardson, though, says the ICC can ensure the best do turn up.
"To some extent you can control that," he said.
"In the participation agreements there is a clause that all the best available players must be selected."
Richardson, a qualified lawyer, said the IOC had not said whether a sport from the present roster would have to be removed if cricket was to be admitted.
"When taking any decison to admit any new sport the IOC has to take into account a limitation on the number of athletes," he said.
"Cricket, being a team sport, could not cater for more than six to eight teams and we must not send a beach cricket team or a six a side team. It must be in a format played at international level."
Richardson, who said there would be regional qualifying and finals and then an Olympic qualifier, said Twenty20 would be the perfect fit.
"Twenty20 is the ideal format and would be even better than (rugby) sevens, we would say, as it is one of the mainstream formats of cricket," he said.
Richardson doesn't believe the fact neither the West Indies nor England could send a team would be a problem.
"The countries who make up the West Indies haven't raised it as an issue and similarly England, well they could include a Scotsman and an Irishman.
"I don't think that it will be an issue.
"As there are only going to be six or eight teams it is unlikely teams like Barbados and Jamaica would qualify."