JOHANNESBURG (AP) — Church officials prevented rescuers from freeing victims at the site of a six-story building collapse, causing a loss of lives, Nigeria's emergency agency said Friday.

Most victims were South African, according to the South African government, which said at least 67 South Africans died and 17 appear to be missing in the rubble of the Lagos building that had a shopping mall on the ground floor and guest rooms above.

The building at televangelist T.B. Joshua's Synagogue, Church of All Nations, collapsed at 12:44 p.m. on Friday but rescuers did not get full access until Sunday after 5 p.m., Ibrahim Farinloye, spokesman for the Nigerian Emergency Management Agency told The Associated Press.

He said rescue workers were on the scene by 1:50 p.m. Friday and they saved three people. But church officials prevented rescuers from getting further access, he said.

He said rescuers lost the two or three critical hours immediately after a disaster when most lives are saved.

"Some of us were even attacked (by church members). We wanted to ask the military to deploy to force access for us, but there were many foreigners at the church and we did not want to create an international incident," Farinloye said in a telephone interview.

Church members also were hostile to journalists and smashed at least one television camera, he said.

Rescue workers were only allowed to get to work after Lagos state Gov. Babatunde Fashola met Sunday at the disaster site with Joshua and threatened to take over the site if rescuers were not allowed to do their job, said Farinloye.

Eighty bodies were recovered and 131 survivors were found by the time the rescue operation ended at 3:35 p.m. Thursday, Farinloye said.

Joshua could not immediately be reached for comment but his church published a denial on Facebook, saying: "It is very sad that inaccurate reports are coming from some quarters that we are not cooperating with the rescue teams and other agencies."

A South African team that arrived in Lagos on Thursday will be involved in recovering bodies, South African Cabinet Minister Jeff Radebe, head of an inter-ministerial committee on the collapse, told a news conference Friday. The team believes 84 South Africans died there, implying not all bodies were recovered.

Radebe did not respond to a reporter who asked why Nigeria's government has made no comment on the tragedy.

The process of identifying victims will be lengthy and might involve getting fingerprints and DNA from family members, South African Foreign Affairs spokesman Clayson Monyela told the AP.

"I think there will have to be a further investigation into what happened and how the church responded, but for now our focus is on helping families identify bodies and see that the injured are attended to," Monyela said.