you are right. try jump start first.

either from your friends, or AAA or your insurance service( if your insurance includes road assist)
if the car can start with jump start, it normally is battery, but it does not necessarily mean you need to replace the battery yet. you could just recharge it first to see how it goes.