I can see that you are coming from a good place with this sentiment; however, I disagree with some. Yes, speakers invest their own time and travel money (usually not reimbursed) to do these events but let us not pretend there aren't other reasons for this. Many are consultants marketing their name brand. Others simply do it because they like it and are willing to pay the price to be in that position. To be honest, some sessions are not very good. They need to know that in no uncertain terms. I have heard this sentiment from an organizer one time and was floored - I dropped those evals (for sessions I attend) in the garbage like it was hot. Do not waste my time. 

I started speaking at SQL Saturday's last year. I love it when I get an interrupter. It tells me they are engaged. They point out things I didn't think of and I can usually direct the conversation to a close as I see fit - it is after all my talk. I do not let things escalate to the point where it is unproductive. 

>>Feedback should be constructive and positive in every way, as you said, actionable and not an outlet of the attendees mood on the day.
Here is my perspective as a speaker: I am not there to be told how great I am. The whole point of feedback from the audience is to improve and connect more with them. I do not want to hear positive feedback. I want to know the problems and things that didn't go well. I don't fly across the country to be placated because it sounds nice and positive. Incumbent with speaking is being able to work a crowd and manage them. I'm quick on my feet so that helps but it also let's me hone other soft skills.

I used to be a debater in high school FWIW. That has shaped me. I have found that it does not matter if you are presenting in front of academics, church, social event, professional trade group, people at the bar, etc - same principles apply.