Taiwan Youth Hostel Association has a network of hostels around Taiwan. A complete list of hostels in English can be found here.
To promote travel by young people the Taiwan Youth Commission has set up a Youth Guesthouse Network. It provides accommodation at government hostels for NT$500 or less on Sunday to Thursday. You need to have a foreign passport and be aged 15-30 to take advantage of the offer. More information can be found here.
There are also many hotels around Taiwan. The area around the railway station in most cities usually has a number of hotels. Prices and standards vary, but it shouldn't be too hard to find a room for less than NT$1,000. These hotels often have signs in Chinese only, so if you can't read Chinese ask someone for directions.
Check out the following budget hostels around Taiwan
Kaohsiung City - Gaoxiong (高雄)
Hualian (花蓮)
Taidong (台東)
If you can't find what you're looking for here. Try a comprehensive hotel search for Taiwan .