Dodgy tertiary education businesses have been given unknowing TAFE students a debt without their knowledge and delivering poor educational results.

Vocational Education Minister Michaelia Cash says the ombudsman has been receiving complaints and it is believed 25000 students to date have claims totalling around $50 million.

The scheme introduced in 2012 by the Labor government, was later replaced by a tighter regulated loans program which Ms Cash assures will keep dodgy businesses out.