As you may need to come back in the future. If I were you, I probably would go but talk nicely with the current HR or the hiring manager. Let them know you like to work for them but 20K is something you could not ignore. They are human too and they should understand. Then you do your best to minimize the impact. My thought is try your best not burning the bridge as it is a small world. I felt you can still go for the money.