Being harassed by a gym instructor

A while before I got married I used to train at a gym. The instructor, a male, had a lot of other males and females in this class. At the time I knew that training in a mixed setting was haraam but I was ignorant and am still trying to repent for this sin. I stopped training after 3 months and then got married 2 years later. Now, after 2 years, the instructor keeps harassing me and making false accusations of me having an affair with him. All this time before I got married I had no contact with the gym or trainer and did not even hear from this trainer. Alhumdulillah, my husband can see that this is nothing but harassment and that I did not commit that sin. I however feel as though I did commit the sin by just being in the presence of men in a mixed gathering and training. He is accusing me of a whole load of false accusations which I did not commit and I have tried to get legal help but the more I fight back the more out of control this situation becomes. I am really afraid that people are going to find out and assume the worst even though I did not have an affair. Please can you advise me accordingly.