Are you sure about the "normal people" part? I mean, that group of speakers includes me, so...
I believe this stuff, communication with humans, is just hard. We all struggle. Fact is, sometimes, we just don't notice that we excluded people. It's not that anyone sets out to. Just the opposite. But you see someone you know well, but you haven't seen in a while, so you run up, hugging & smiling & talking. Yeah, the person opposite, that you don't know at all, may feel excluded. I don't know how to fix that.
However, what I do know is, I welcomed every single "First Timer" I saw. I welcomed, individually, all the attendees to the first timer event. I work, hard, on being as approachable as I can be. I try to embrace the whole "aggressively friendly" attitude set by Kevin Kline.
We really are just humans, with all the good & bad that implies.