Recently we released the results of the annual survey to determine interests for the 2010 Summit. That’s a step in the right direction, sharing the results, but what we didn’t do was share what we thought we had learned from the results, and we were taken to task some by Steve Jones and Brent Ozar (great comments on this one) for not doing that.
It’s a case of us failing by doing more. If we had not released the results, we wouldn’t get in trouble for not interpreting the results! Speaking only for me here, we missed an opportunity, but it’s a good lesson to learn. I sent the following to the Board as thoughts around this:
- Are we asking questions that will give us answers that are useful. For example, now that we know backup compression is popular, what do we do with that info? Do we schedule multiple sessions on it? Make sure we have at least one session on it?
- Bias check. Are we tilting the questions in the direction we want to go (entirely human, and often done without realizing)
- Get buy-in from PASSCab. Politically it’s valuable to say we asked for input, realistically we may get some good feedback to make it better
- Release our eval with the results, should include actions we plan to take, areas of ambiguity, lessons learned for next time, and potentially the need for a follow up to clarify areas we didn’t get right on the first try.
Note: PASSCab is our advisory group, a chance for us to get some opinions from outside the bubble of the Board – see the post by Tom for more details.
We’ve got a lot of learning to do and process to build around surveys – or do we? Part of me thinks absolutely, we need to go through the 10 step process and follow it perfectly, lest we be criticized. If we can spend a few hours to do that and still field a new survey quickly, that’s good. If every survey gets bogged down in hair splitting review, is it worth that to get perfect answers?Surveys that will get reused merit more investment, and probably all surveys benefit from a simple process, but – see how it easy it is to add overhead?
I’ve got another post coming up next week that talks about the criticism side – might surprise you with some thoughts there.
See this post by Tom LaRock on the same issue.
SQLAndy 
Knowing how these surveys originate in PASS, I think it would be an interesting undertaking to determine if its worth the time to go through every survey before, during, and after for “analysis”. That could bog everything down to an infinite degree.
This particular survey was/is released every year, its always linked from the speaker resource page and analysis isnt & shouldnt be done at the PASS level in my opinion because the point of the survey is to show the community speakers what the community wants to see. Its up to the speakers themselves to decide how to interpret the content of the survey, otherwise if a speaker submits an abstract on XYZ that PASS interpreted as important based on the survey and then subsequently they arent selected, youd hear complaints about that as well.
Its also important for the members of the PASS community to know that the survey results are used by the abstract selection teams as part of their process in selecting abstracts. As with all things in these teams and the selections though, there are more moving parts than simply 1 surveys results.
LikeLike
Allen, that makes some sense, but think if it’s targetted at giving speakers the right info maybe it needs some work, or at least annual input from a subset of speakers. The small sample size worries me.
I’d be interested in hearing more about how it’s used by the selection teams.
LikeLike
We would also like to mention that most of those who find themselves without having health insurance can be students, self-employed and those that are out of work. More than half with the uninsured are really under the age of Thirty-five. They do not experience they are requiring health insurance because they are young plus healthy. Its income is frequently spent on property, food, and also entertainment. Some people that do represent the working class either complete or part-time are not presented insurance through their jobs so they go without due to the rising cost of health insurance in the usa. Thanks for the concepts you talk about through this website.
LikeLike