I got an email recently that notified me that session feedback from the Summit was available for my talk. I’d had a lot of people in the room, and was curious how things went. I think the session was OK, a little off on time, a few too many questions I tried to answer, and perhaps a bumpy flow.

However when I got my scores, I had a 2.85 for the session overall, with various aspects of the talk being rated from 2.5 to 2.9.

Well, I sucked.

That was my first thought. I’ve been getting rated, and evaluating speakers on a 5 point scale for quite a few years at PASS events. I was surprised, and disappointed, and then a bit embarrassed that I hadn’t delivered a good talk at the Summit. Since I hadn’t delivered that talk in public anywhere prior to the conference, I thought I had made a big mistake. Apparently my practice that week in my hotel room had been for naught.

However then I saw this note in another email: One of the changes this year was to move from a five point rating scale to a three-point scale.

Hmmm, I missed that in my email somewhere, and didn’t notice this as I filled out a couple of session evaluations.

I don’t think there’s anything wrong with changing the scale. Personally I like the 3 point scale, but it wasn’t a change I noticed. The first communication with speaker feedback didn’t mention this.

Scale matters. Many of us know that by manipulating scales, we can make data look different, We can prove a point that might not be supported by a different presentation of the data on another scale.

Our clients and business users come to know and expect the various ways we present data. They will start to internalize scales and interpret data based on their expectations. We can change scales, but we need to make it clear and visible that we have changed scales.

Personally I would have appreciated the results being reported as:

Overall Session Score: 2.85/3.00

instead of

Overall Session Score: 2.85

That little extra information can mean a lot. Keep this in mind as you make fundamental changes to the way you present data.

Filed under: Blog Tagged: data analysis, speaking, syndicated