The home affairs select committee said it was “not acceptable” that some were being detained for up to two months.
Chairman Keith Vaz said the children had “done nothing wrong” and should only ever be held as “a last resort”.
The government said treating children with “care and compassion” was a priority for the UK Border Agency.
The committee’s report says that nearly 1,000 children a year are detained in the UK while they and their families await removal from the country.
On average, they spend more than a fortnight in detention, although periods of up to 61 days are not uncommon, it says.
Earlier this year, children’s commissioner for England Sir Al Aynsley-Green said the practice of holding children in detention should be ended altogether.
‘Unlikely to abscond’
Keith Vaz: “We felt the detention of children was really a last resort”
Mr Vaz referred to Yarl’s Wood detention centre, in Bedfordshire, in particular, which has been heavily criticised in the past.
He said that despite some recent improvements, such as a purpose-built school, it remained “essentially a prison” and “no place for a child”.
The committee said it was difficult to justify detaining families when they were very unlikely to abscond and in future the UK Border Agency should consider other alternatives such as electronic tagging.
“It is not acceptable that we are detaining so many children for such long periods of time – these children have done nothing wrong, they should not be being punished,” Mr Vaz said.
“It must always be absolutely the last resort to keep a child detained for any length of time.”
Katherine Hill, from the Children’s Society, said children should be kept in the community and there was “little chance” of them absconding if they were registered with doctors and schools.
“There are examples from abroad where this has worked very effectively – and actually the number of successful returns is far higher at the end of the day,” she said.
‘Frivolous appeals’
Even relatively brief periods of time spent in immigration detention can damage a child’s mental and physical health
Amanda Shah
Bail for Immigration Detainees
The committee also called for reform of the asylum legal process and pointed out that 90% of appeals against deportation were never heard.
“The system must be cleaned up so that those who have been refused settlement in the UK are deported as soon as possible and only detained as a last resort,” Mr Vaz said.
Local authorities must also do more to protect the wellbeing of detained children, he added.
Immigration minister Phil Woolas said: “When the independent courts find a family has no need for protection, we expect them to return home rather than putting in frivolous appeals that clog the legal system.
“We prefer families return home under their own steam – enforced removals are very much a last resort.”
Amanda Shah, from Bail for Immigration Detainees, a charity which helps represent families in Yarl’s Wood, said the report’s recommendations did not go far enough.
“In particular, we disagree with the committee’s acceptance of the detention of children for short periods of time, particularly as research released last month showed that even relatively brief periods of time spent in immigration detention can damage a child’s mental and physical health.”