Thanks everybody.
I've thought about what you wrote and it helps. I know its possible to misunderstand what people mean. They are being nice but you take it the wrong way. I'm now thinking that this man thought he was being friendly but the way that he said it was wrong as it came out as sounding dominating. There are people who are like that and they cant help themselves. But I wish I could stop being effected by people dominating me. Maybe I did the right thing and said nothing and made him feel ignored.