My ex won't let me settle with another man

Three years ago, I met a man who I fell in love with.

One thing led to another and I got pregnant and had a baby, who is two years old. However, I was forced to leave the father of my son because he used to mistreat me.

I'm now involved with another man, who accepted my son and I. My new boyfriend is good to my son and I.

However, the biological father of my son keeps on threatening us that he will not let us live together. My son's father knows where we live.

I do not want anything to happen to my new boyfriend. I'm so scared. What should I do?