Immigration Act Article 12

A national without household registration in the Taiwan Area who has entered the State with a foreign passport or stateless travel document shall exit the State with the foreign passport or the stateless travel document and may not apply for residence or registered permanent residence unless he/she meets the requirements of Subparagraph 3, Paragraph 1 of Article 9 or Subparagraph 2 or 4, Paragraph 1 of Article 10.
中文原文 入出國及移民法第12條
臺灣地區無戶籍國民持憑外國護照或無國籍旅行證件入國者,除合於第九條第一項第三款或第十條第一項第二款、第四款情形者外,應持憑外國護照或無國籍旅行證件出國,不得申請居留或定居。

English translation for reference only; the Chinese text prevails. Not legal advice.