What's your current base for GC?
You said you have not been working, so it's not LC based. (if it is, you may have trouble to survive RFE.)
If you are based on self-petitioned I-140 or family based I-130(adult son or sibling), don't worry. You are forever eligible, just file I-485 or CP whenever you are ready to return to USA in the future.
If your basis is LC or employer sponsor, you need to start over if you abandon I-485 this time.