It's not always necessary to

捉奸捉雙, if it comes to courts and he denies it under orth (assuming you are right), he'd be commiting perjury.

He's committed adultry, depending on states, you can ask for sole custody, and probably will get it.

A traitor will always be a traitor, he's done it already, and once given opportunity, he'll do it again. If you divorce him, there will be pain, but it'll be temporary; if you stay with him, there will also be pain, and it'll last, and he'll probably do it again.

I'd suggest marriage counseling as first step if you have doubts about divorcing him. But personally, I'd divorce him outright.