When you've passed that point of non-awkwardness

You know how it's generally accepted that within the first few times of meeting someone, it's okay for you to forget their name and after that it's just becomes awkward? I have a couple of situations that aren't that, but that are similar. The first: I may have written about it, but I don't think so. There was a man who was regularly on my bus who behaved as if he knew me. Well, not quite me, but my mother. He knew my mother and used to talk to her. The first couple of times it didn't seem infeasible even though I couldn't figure out how he knew me. Then he started talking about not having seen her on the bus lately. I mentioned that my mother retired but never took the bus. He replied with, "maybe not often, but she did." Hmmm. Then more and more things weren't making sense and I realized that there was no chance that I was (or my mother was) who he thought. But how do you bring that up after talking to someone for months? So one day, when he mentioned my mother (we had progressed to talking about ourselves), I 'fessed up. When his denials that it WAS my mother didn't work, he stopped talking to me. This man I had spoken to for months wouldn't even make eye contact with me. One day I ran into him, a friend of his, and his kid at the park. I started talking to his friend...it became impossible for him to ignore me so we re-established eye contact and "good morning" but never moved past that. I've never been to happy to see a virtual stranger move to a different home on a different bus route. The current: A couple of years ago, a man included me in the list of family names when they were making their Christmas plans. I figured that someone would correct it or he would realize it. It was only a couple of emails, so I ignored them. Then a few months later, I received family pictures, to the same group of addresses and all the "reply all" responses. But I didn't know how to respond. So I didn't. There's still the occasional email but nobody ever mentions me not responding, and it's just past the point where I feel I can say something. I feel like it would be inappropriate to turn down the invitation to the football game I just got an evite for. Right?