you can use the former co-worker's letter

You don't have to use the manager's letter, you can use co-worker's letters, find one that has good relationship with you. Normally if you have stong case, the DOL will not contact them. If they want to audit, they will mainly audit your current sponsor.

For example, I worked for a Fortune 100 before, but it merged with another company and the boss left the company already, so I asked my former co-workers to issue me a letter, no problem at all. My perm was approved in 2 days.