It won't catch fire

If it was spilled oil during oil change on the exhaust pipe, it should go away soon. However, if it keeps smoking after a few days, you should check for leak. (in the latter case, it could be that the oil filter was not tightened properly)