What do I do about him I am confused?

Well, I'd say you will never know unless you ask him about it. Otherwise, you might never know what could have been.