However, Firth said it would be a tragedy if the charity's work came to a halt.
Cafod director Chris Bain said the charity had been unaware of the allegations until it was contacted recently by The Times newspaper.
In a sign of the seriousness of the abuse allegations against Oxfam, she will meet the director general of the organised crime agency today to probe how it can better implement laws on sexual exploitation and abuse in the sector.
She said: "The recent revelations about Oxfam - not exclusively the actions perpetrated by a number of those staff - but the way the organisation responded to those events, should be a wake-up call to the sector".
"I think now, for the thousands of people that volunteer, that donate to Oxfam, for their aid workers even today that are working in very hard circumstances around the world, he owes it to them basically to resign and enable Oxfam to look to the future, to focus on their core activities of poverty reduction and alleviating suffering...and I think that requires a change of leadership".
A DfID spokesman said: "We often work with organisations in chaotic and hard circumstances. Devastated for the women who were used by people sent there to help them, devastated by the response of an organisation that I have been raising awareness for since I was 9 years old #oxfamscandal".
They have also let down the vast majority of individuals working in humanitarian relief who do so with "great passion and integrity", OSCR said.
Oxfam has neither confirmed nor denied that specific account but has said an internal investigation in 2011 had confirmed sexual misconduct had occurred and has apologized.
The Times of London reported last Friday that young sex workers were hired by senior staff in Haiti after the natural disaster which devastated the poor Caribbean country and left up to 300,000 people dead.
Van Hauwermeiren has also written an open letter in which he says: "I'm not a saint, I'm a man of flesh and blood and have made mistakes (not easy to admit), and I am deeply ashamed".
On Sunday, the charity was hit with similar allegations saying that Oxfam staff had paid women for sex in Chad in 2006.
The statement in full: "We have identified that Gurpreet Singh who was one of those dismissed by Oxfam as a result of the sexual misconduct case in Haiti in 2011, was subsequently hired by Oxfam as a consultant In Ethiopia from October - December 2011".