You are supposed to file tax for these three years. If you are H1B from P.R. of China, there is no treaty that allows you federal income tax free.
In 2003, you may be able to file 1040NR due to the fact you are in US less than 183 days. In 2004 and 2005, you should be consided as Resident Alien for the tax purpose, you will need to file 1040.
Since you did not have tax withholding, you are going to owe a lot of taxes. I suggest you consult a qualified tax professional to file these reurns and pay the tax. IRS has no statute limitation(time limitation) to after these tax since you did not file the returns.
You will also owe state tax since state never recognize any federal income tax treaty.
-Zhicheng Lai
Enrolled Agent in MD