It is good idea to visit police station. Although you cannot make a formal report to let police start anything, you can discuss with police about your worry. The good points are:
(1) More or less, police can give you some useful words/advice.
(2) He has been a trouble for you. In the future, you don’t know how big this trouble will become. So if at the beginning, you have a record that you have ever visited police station for his threatening, this record possibly is useful when something happened.
