Police in the US state of New Mexico have rescued 11 malnourished children kept in a filthy and heavily armed remote desert compound.

The children, aged one to 15, were so famished they "looked like third world country refugees," Taos county Sheriff Jerry Hogrefe said on Sunday, reports CNN.

They were kept in an underground trailer, hidden from the outside world.

"The only food we saw were a few potatoes and a box of rice in the filthy trailer," Hogrefe said.

"But what was most surprising and heartbreaking was when the team located a total of five adults and 11 children that looked like third-world country refugees not only with no food or fresh water, but with no shoes, personal hygiene and basically dirty rags for clothing."

Authorities were tipped off to the scene in Amalia, near the Colorado border, after someone forwarded a message believed to be from a third party. 

The message said: "We are starving and need food and water."

After executing a search warrant on August 3, officers found the children, along with three women believed to be their mothers who were taken into custody for questioning and were released, officers said.

Authorities also arrested two heavily armed men -- Lucas Morten and Siraj Wahhaj -- at the scene, the sheriff's office said.

An AR-15 rifle, loaded 30-round magazines, four loaded pistols and many rounds of ammo were found in the makeshift compound that included a small travel trailer buried in the ground covered by plastic with no water, plumbing, or electricity, CNN reported.

Wahhaj was wanted for the abduction of his three-year-old son Abdul Ghani Wahhaj in Georgia, authorities said.