sometimes one thinks too much and then

does not know how to behave. Like said above, if you enjoy going with them, just say may I join you? If not just say you guys have fun. Do not over think, regardless if they ask you or not. If you start to think about it, your facial expression will not be natural, and they will feel awkward as well. Take it easy, just for fun. If not fun, no need to join.