Category Archives: SQL Community

A Few More Thoughts on Summit Session Selection & Growing Speakers

Last week ended up better than expected when PASS decided to release feedback to speakers (selected or not) who requested it. I imagine many did and the feedback on the feedback was surprisingly good – and that is good. That’s a good step and I hope it will be become standard to deliver that feedback to anyone who submits a session. The next step is to provide the same level of feedback on the speaker evaluation. Right now we don’t know what matters or doesn’t, and there is no reason for that. If people can work hard and improve their standing, that’s never going to be bad for PASS except in one way – it will make the final choice of the sessions for the schedule even harder!

If you haven’t already, I encourage you to read How Conference Organizers Pick Sessions by Brent Ozar, a great write up. Building a schedule worthy of a Summit size/cost event is non-trivial.

I also wonder if there hasn’t been an interesting dynamic in past years where those that (arguably) wanted/needed feedback the most were the ones not selected, and those are the people (like me, last year) who are apt to not make much of a fuss.

I hope the program review committee will include some outside voices, and that it will start soon, and that it will do transparently. We need to see the discussion and the challenges.

Changing focus to growing speakers, Chapters and SQLSaturday (and maybe 24HOP) are the farm club that feeds the Summit, and I think do so in grand fashion, but neither have anything close to the demands/process that go into the Summit selection. We (me) had hoped that SQLRally would be the piece that plugged that gap. With the abandoning of SQLRally in the US I haven’t seen much in the way of thinking about whether the problem remains or a way to fix it. I think it does, and I still think “regional” events are the key, something that is more selective than our Chapter/SQLSaturday events, but not as selective as the Summit. Or maybe exactly as selective as the Summit, but with the same idea we had for SQLRally, largely exclude any previous year Summit speaker so that “new” speakers can get a chance to grow.

I’d like to see regional events, but I don’t know that PASS (the Board) has the passion for them, so how else can we do it? Trained evaluators are one way, and we’ve seen the beginnings of that with well known people in the community offering to review abstracts. We could charter a program to train people to do abstract and presentation evaluations in a detailed way (somewhere back in the archives I have a 60 point eval sheet I suggested at one point) and as many (if not all) would be speakers, they could do one here and there as they attend events if someone has requested (and is ready for) an in-person evaluation. Maybe there is an opportunity to do some regional classes where experienced speakers attend for a day or two for instruction and then evaluation by the instructor – I’d have PASS pay the instructor and the overhead, make it free for qualified attendees. That’s something we could easily trial at the Summit. I think how great it would be to have the discussion about what we’d teach. Maybe it can be done online. Maybe there are other ways?

We’ve addressed the quantity problem of speakers, though I think we have to continue our efforts to find more people to step to the front of the room. What we need now is a deeper focus on quality. That would not just lift the Summit, but all the events that tree up to it.

I was just thinking that we hold meetings each year at the Summit for Chapter Leaders as a place to hear and share ideas, and we do the same for SQLSaturday leaders. Why don’t we have something for the speakers? Is it a different dynamic, or a so far missed opportunity? In the early days of SQLSaturday we knew that to grow we had to find that one person in each city that would take on the huge task of putting an event together, but we also saw that the entire success of the eco-system was based on speakers. That’s something I’m reminded of every time speakers sign up for SQLSaturday Orlando and fly in from some other city to attend. I wonder if that lesson hasn’t been lost a bit at the Board level, and I’d suggest that maybe it’s time to have a portfolio that focuses on growth of speakers.

I’m not arguing for doing more for the sake of doing more. There are ways here we can serve our members, directly and indirectly, just be putting some more effort into the growth and education of our speakers.

Project Moriarty

This past weekend I started writing notes for what I’m calling Project Moriarty, what I hope will be five questions in a row that have to be solved individually, but then based on information in the questions and correct answers will lead those that complete the challenge to solve the overall challenge posed by Moriarty. It’s a leap from what I’ve done before and one that is a large unknown, and it has the feeling of stacking dominos, if I blow one question in the sequence I could ruin or invalidate the ending. Just writing my notes I’m undecided, is it five questions, or six? Can I build the clues I will need in four days to get to the summary on day 5, or perhaps include some final ones in day 5? Having a separate question on day 6 seems anti-climactic, but we’ll see.

I have no idea how long it will take. It seems like if I can nail down the plot, it might go quickly. Of course “nailing down the plot” isn’t trivial! It’s also a fill-in project for what little time I have left after working on SQLSaturday Orlando, doing some other presentations, prepping for the Summit, etc, etc. I’m hoping to finish by mid August, but it will take what it takes.

Feedback on My Abstracts

Here’s the raw feedback from my submissions for the Summit. To those who wrote the feedback for me (and all the rest), I say nicely done. As much as I’d wish for still more feedback, this is useful. The only complaint I have is about “contractions” being a rule that is, as far as I know, unpublished. I look at this and think, why didn’t we do it years ago? Agree or disagree with the comments, anyone wanting to do better/increase their chances will find a lot of value in the feedback.

 

 

Andy

Warren

Don’t Be The Next Target – How To Secure Credit Card Data

The first sentence in the abstract needs the word “data” inserted between “card” and “now”.  Unique topic, but I am not sure it will attract many attendees.

Andy

Warren

Don’t Be The Next Target – How To Secure Credit Card Data

The abstract is well written and appears to cover the topic of securing Credit Card data very well.  However the lack of any demo is what has me lower the abstract rating by a point and subjectively by 2.

Andy

Warren

Turbocharge Your Career with A Learning Plan

Good topic. Good consistency across session name, abstract, topic and goals. I personally attended a session on the same topic during a SQL Saturday event and attendance / interest were pretty good. However the lack of practical examples and/or demos might make pretty hard to keep the attention for 75 minutes on this topic which is supposed to be more practical than academic.

Andy

Warren

Turbocharge Your Career with A Learning Plan

Excellent abstract. Clear information, gives a great narrative, and makes for compelling reading.

Andy

Warren

Turbocharge Your Career with A Learning Plan

Contractions.

Andy

Warren

Turbocharge Your Career with A Learning Plan

The abstract does not explain the why/how of the session.

Andy

Warren

SQLChecker – Moving Beyond Policy Based Management

Solid abstract

Andy

Warren

How To Write a Question Of the Day

Well structured abstract and extremely interesting topic. Just a minor concern regarding the fact that no practical examples will be provided as naturally expected on this topic and especially on a 10 minutes time window.

Andy

Warren

How To Write a Question Of the Day

This appears to be very SQLServerCentral centric, after all a question of the day is not something that shows up everywhere. Not sure that given this that it’s quite right as it’s pushing that site.

Andy

Warren

How To Write a Question Of the Day

Contractions

Andy

Warren

Welcoming The New Team Member

The 1st half of the abstract is perhaps overlapping with some typical HR duties. Also, unfortunately, in big corporates, the logical separation between contractors and permanent staff is enforced as part of the top down cultural approaches. Although the principles in the abstract are absolutely valid, perhaps the context of track / event might not be the best to discuss this topic.

Andy

Warren

Welcoming The New Team Member

Nice abstract, leaves wanting more without being linkbaity.

The Selection Process

According to this post from Amy Lewis (PASS Board of Directors) there were 943 abstracts submitted for the 2014 PASS Summit by 335 speakers competing for 144 slots on the schedule. An abundance of riches to be sure, but it also marks a time of expectation, exultation, disappointment, and even disenchantment as some win (get selected) and some lose (not selected). Those not selected are certainly not “losers”, but there is a sense of having failed in some way. Last year was the first year I wasn’t selected since I submitted my first abstract for the Denver event back in 2000 (I think) and I hated being “left out”. It’s not as easy to accept, or understand, if it’s your first time submitting, or if you’ve never been selected, because the feedback is minimalistic. Track full, or duplicate session, that’s all you get. There is also a tendency to wonder if sponsor reps get special treatment, or Board members, or people on some secret “A” list. Given the lack of feedback, I don’t blame anyone – everyone – for wishing there was more feedback and more transparency.

Before I get into my thoughts on the Summit selection, I’d like to talk about SQLSaturday selection. Here in Orlando we have what hardly anyone would call a process, just some guidelines. We prefer local speakers and work hard to get them, we try not to turn anyone down (which is why we typically run 8 tracks, and we love the value and diversity of speakers that fly across the country to donate their time to our community. We do what we can to help them be successful. We ask them to be servant leaders and volunteers by serving lunch to our attendees each year (while wearing a SQLSaturday chef hat or apron of course!). We’re so lucky that we don’t have to send out “rejection letters”, at least so far. If we get in a jam Kendal or Rodney or Bradley or Shawn or myself will give up a slot to make it all work out. We build our schedule trying to get that magical and mythical mix of skill level and topic and approach, choosing one of the 2 or 3 abstracts that most speakers submit, unless they stated a preference for one of them. Even with the given of putting everyone on the schedule, building a good schedule is hard! We sweat over it, and that’s with 50 or 60 abstracts. Imagine having 943. We also try to do one or two paid seminars each year. We’ve talked about building a process for that, but it’s still just guidelines, and mostly unpublished. We’re looking for a topic that will draw enough attendees to cover the costs and send the speaker home with a little cash, we want a speaker that we really think can deliver on the promise of the abstract. We don’t always take the “best” speaker or abstract. We like the idea of helping others grow and get ready for a Summit seminar some day. We will sometimes pick a speaker/topic and then ask for tweaks to the title and/or abstract that we need are needed to help us market it effectively. Entirely analog, not something where we score sessions and speakers, though sometimes I think we’d benefit from doing so, just to help us think it through a little better.

Now let’s return to the Summit selection process.

What’s the best way to build a schedule for a top-of-the-pyramid paid event? Is it the detailed and heavy scoring and weighting process we seem to have now? An all out election by the people as advocated by my friend Brian Kelley? A combination of both (we used to have “community picks”, what happened to that?)? Beyond that, should there be any other rules? The current plan limits speakers to no more than four abstracts for the main conference. Should we consider (as I have argued for a few times) a rule that requires speakers to sit out a year after being selected? A rule to limit speakers to a max of one session?

Good questions I think. Looking at it a different way, what would I want to achieve?

  • A schedule that has top tier presenters on it, both for the value of what they deliver and for the marketing value of having them participate
  • A schedule that has something for different skill levels and interests, ideally with two tracks per “major” focus area so that not only is there content, there is always a great “plan b” if the first pick room is filled to capacity, or just not a perfect fit
  • A process for building the schedule that is as clear and as consistent as I can make it, explained before the abstracts are submitted (which is also the time to specify any topics of special interest or disinterest each year)
  • A process for penalizing speakers who fail to show, or get really bad ratings, or break the speaker agreement in some way
  • A process for providing some kind of feedback about why their session was, or wasn’t, accepted
  • A process that volunteers can take and own and execute, without fear of the backlash on the day the schedule is announced
  • A process that allows for analogy input because we’ll never have a perfect formula
  • A process for dealing with grievances (from the committee, or the speakers)
  • A process that considered the speaking experience (including evaluations and recommendations) from other PASS events
  • A process that considered previous/estimated interest in the topic
  • A process that speakers understood, perceived as fair, and that gave everyone a chance at making it to the show (but not necessarily an equal chance)

You might not agree with all of those and I probably missed a few too. I think about what I’d want for a paid event, in particular one that is the only fund raiser I have, and I don’t think I could bet it all on an election. Maybe that’s wrong, but I think there is real value in a team looking at options and making hard decisions about what gets on the schedule. I do think we should return to having some of the sessions be selected by the community, it’s a great way to make sure new voices get heard, to correct minor flaws in that years process, and to engage the speakers/community in a lively and interesting way.

The problem is today that we don’t really understand the process. Maybe it sucks. Maybe it’s really good and we just don’t see it. It seems to consist of two parallel threads, where one team does a blind evaluation of the abstract and another team evaluates the speakers. Do typos matter? Does length of abstract matter? What goes into evaluating a speaker? How are conflicts resolved when two people submit abstracts in the same niche? Can someone at the manager level override the scoring and mandate someone be put on the schedule? Is there some process for removing people that are anti-PASS? Do former Board members get preferential treatment? Do current Board members?  Sponsors? Publishing details of the process and the results of the process would go a long way towards stamping out any bit of distrust in the process. Of course transparency can also provoke a lot of arguments and maybe hurt some feelings too, and that’s worthy of consideration. Would you want the whole world to see your abstract rejected as “numerous typos, bad grammar, no focus” or you being rejected as “previous bad conduct” or “consistent low eval scores”?

Speaking of transparency, this is what we used to post – why don’t we do that now?

 

image

 

image

image

 

 

It’s easy to forget that our process is run by volunteers. Amy Lewis is on the PASS Board and is a volunteer, and she was helped by Melissa Coates and Lance Harra as Program Managers. They may not have arrived at the perfect answer, but I don’t doubt for a moment that they labored hard to follow their process and deliver sessions that would be worthy of the Summit. Much as in my comments earlier today about members of the Board being eligible (or not) to give paid seminars, I’d like to see us have a process where a volunteer can do the work we ask and not have to take any flack about the results, at least on a high level. We can’t keep doing it the way we are without a lot of volunteers, no one wants to volunteer just to be yelled at, or to have someone imply they somehow played favorites. We don’t do a very good job of celebrating their efforts, something else we should work on.

While I’m writing, I also want to mention that when I was on the Board I don’t think we ever talked about the selection process as far as who was or wasn’t speaking. Allen Kinsel was working on tools and we had an idea it was going on, but I’m sure we never as a group were involved in any decision about selecting or not selecting anyone. I think that’s good, but I also remember feeling strangely distant from that process, and really the running of the Summit as a whole.

I don’t think the current process is bad, but I think we can do better. Here are the changes I’d like to see discussed this year for implementation next year:

  • PASS to publish a video and documentation about the entire process as part of the call for speakers launch. Let’s make sure we all understand the rules, and let’s leverage those speakers in the community that have spent a lot of effort figuring out how to write great abstracts and get great scores and deliver great presentations.
  • Commit to providing every speaker/abstract with feedback privately, with the speaker having the option to share it publicly (a step towards full transparency)
  • Limit speakers to one session (max opportunities)
  • Require x percent first time Summit speakers each year, provided there are enough candidates to hit that goal and they can document speaking experience (I like the idea of anyone new to the Summit requiring 2 references from other speakers)
  • Limit speakers to presenting every other year
  • Re-establish the community vote for some x percent of the available sessions
  • Limit speakers to submitting two abstracts. That would reduce the volume and increase the time spent on abstracts. I think just this one change could be huge in making things better)
  • Require every presentation to have been presented before it is submitted. No more presentations on spec.
  • Someone qualified and authorized to answer questions directly on the day/week the schedule is released, same day answers as much as possible
  • Annual review and suggestions provided back to the Board, with the Board voting each year whether an independent committee is needed to assess changes or not (go/no-go)

I’d like to see similar rules put in place for seminars, but that probably requires extra care on all sides. I’d like to see 100% turnover in that space each year. Because money is involved, it gets complicated.

I don’t claim to have all the answers. I think we try hard to do good and mostly we do. I’m extra sensitive to the needs of the person trying to make it to the schedule for the first time. Let’s help them, even if they end up beating us at our game later on. Let’s try to see both (all sides) and do the things that both build and support trust of the way we do this. We’ll still have some that are excited and some that are disappointed each year, but we can do it in a way that encourages those that didn’t quite make it in to redouble their efforts the next year, and in a way that reminds anyone that made it in already that it’s not a lifetime appointment, we have to re-earn that seat every time. Comment on my ideas, or publish your own. Let’s share ideas and see if we can drive some good changes into the system for next year.

Above Reproach

It’s been a busy few weeks in the world of PASS, lots of good conversations that I’ve followed with interest, in particular this post by Kendal Van Dyke, then follow up posts from Bradley Ball and Andy Leonard, as well as others that were more focused on the overall selection process. Today I’d like to add my thoughts on just one part of those conversations, whether it’s appropriate for a board member to present a seminar (“Pre con”) at the PASS Summit and whether there should also be a restriction on employees of a board member presenting seminars. I’m approaching this as a general discussion, but clearly it’s based on events of this year, with PASS VP Adam Jorgensen (and Pragmatic Works President) presenting a seminar (with co-presenter and PW colleague John Welch) on Monday, then two more by other Pragmatic staffers on Tuesday (Brian Knight, Devin Knight, Bradley Ball, Robert Cain, Jason Strate). I know all  of them, most of them really well, and they are, without exception, good people. Robert is a good friend, John is one of the great minds in our business on the BI/software side, I’ve known Brad since he moved to Florida and we’re both on the SQLSaturday Orlando event team, Devin is great to work with, and I was business partners with Brian for about 10 years. I also served on the PASS Board. That makes this discussion personal, yet it is about business. The trick, if I can manage it, is to talk about issues as I see them and point out some ways we might improve the process without damaging relationships. I suppose I could just not add my voice to the conversation and take no risk, certainly it’s something I’ve thought on, but in the end I choose to have a voice in the community I’ve helped to build.

So, with that long introduction, let’s get started. Here’s what the by-laws say:

 

image

 

The short story is that it’s permissible for a Director to be paid for presenting a seminar with a majority vote of the Board. I’ve read the minutes and so far do not find such a vote, it could have been in the  June meeting for which minutes are not available yet. Assuming that the vote was held and passed, then the by-laws have been upheld. [Related, I was pleased to see a reference in the April 2014 minutes where the Board discussed and voted on Tim Ford participating in SQLConnections, addressing what could be seen as a conflict of interest – that’s good diligence]. If there wasn’t a recorded vote, then we have a problem. Fixable with a vote certainly, but it would cast yet more doubt on the process.

While we’re looking at the by-laws, the only other part that is worth looking at for this issue is this part about employment restrictions, something I voted for when I was on the Board, put in place to guarantee that no company can “take over” the Board or exert an unfair amount of influence. That’s not an issue, right now there are two from Pragmatic on the Board; Adam and Wendy Pastrick. Nothing here to restrict participation of employees of Board members in any way.

 

image

 

I couldn’t find a copy of any guidance to the Program Committee (the group of volunteers that evaluates abstracts and speakers) to see if there were more rules there, and I also couldn’t find the most recent ‘speaker agreement’ that speakers sign when submitting a session to see if there was anything there that might be pertinent. I think most of the latter is about behavior, not any exclusionary criteria for selection. Amy Lewis did post some notes here on the process, but it’s very high level.

Assuming the vote was held, then no rules have been broken. That’s a good start.

Then we get into the details. Are the rules we have ‘good enough’? For general situations involving payment to a Board member I think they are, and I’m not in favor of a by-law change. I do think though that board members have a duty to be seen as being above reproach when it comes to conflicts of interests, and the best way to resolve that is to add some guidance to the program committee (which should be shared with Board candidates) that eliminates Board members from eligibility for any paid pre/post seminar that falls under their charter. I think this protects the Board member and it makes clear to the committee what the rules of the game are, giving them both “political cover” and avoiding the very human temptation to support a fellow volunteer.  The other comment that I have on this is that Mon/Tues of the Summit are busy and valuable days for a Board member, too busy to spend out of touch presenting a seminar – especially for officers. The only downside I see to this is that someone might choose to not run for the Board because of the restriction. Certainly their choice to make, but not one that harms PASS (or them).

Going just a step further, what about main program sessions? There are no current restrictions on Board member participation and I think that’s right. We can’t ask people to run for the Board to give up the very thing that got many of them to the Board in the first place. I think with better transparency of the selection process no additional rules are needed here.

So let’s move on to the question of whether a company that has an employee on the Board should incur restrictions on the participation levels of other employees? Even if we put in place the rule above to exclude Board members from doing paid presentations, they could still be perceived as having lobbied/exerted force to get their people on the schedule. This is really, really tough. Over the years Solid Quality has been in this boat, Scalability Experts, and now Pragmatic. All were successful and hired smart, talented people. Would it hurt their business if they were entirely prohibited from paid seminars at the Summit? I think it could well impact their ability to hire and retain the best. Or, imagine the reverse, a working consultant joins the Board and now the owner of that company finds restrictions being placed on them – do they still support/employ the consultant?

What options do we have? We can rely on process with greater transparency, we can put some kind of rule in place that limits participation by company, or perhaps find yet another option.

For general sessions I think process/transparency is sufficient, but seminars are about money, in some cases a pretty good sized chunk of money, and that means we have to be extra careful. For that reason I think we need to put a hard cap in place and limit any company from having more than two paid seminars. That protects the companies that potentially employ (or are owned by) Board members because the rules apply to all. It’s not perfect, but I think it’s workable, and it’s an easy way to show and guarantee that we’re vested in making sure that participants, and PASS, are seen as being above reproach.

I have no issue with members of Pragmatic applying to do the seminars – they played by the rules. I disagree with Adam submitting one, I think being an officer calls for more care and also requires focusing on PASS business at the Summit, and I fault the Board for approving it (assuming they held the vote).  I don’t think Adam (or his team) went and applied pressure to the committee to be selected, but I do understand that influence is a subtle beast and it could have played a part in the final choices with absolutely no input/contact from Adam (or SolidQ or Scalability in years past). Minus transparency it’s hard to tell, and even with transparency I’d bet it would never be something that was crystal clear, picking sessions – even paid ones – is just very, very subjective (and that’s not bad, it just is).

The rest is up to PASS, and up to us. We can bitch about this year and do nothing, or we can find a way to have a serious conversation and either change the rules, or write down our expectations so that everyone can go into the 2015 selection process  with a clear head. Let’s not have a community where people might not submit because they fear the public perception/backlash because of where they work. This issue is a portion of the larger problem of insufficient transparency on the program committee, and PASS as a whole. Transparency isn’t quite a silver bullet, but it helps us all understand how things work and then we can have a better, more informed conversation if we think changes are needed. How do we get any change done? That’s a good question. My suggestion is that PASS commit by the end of July to a separate committee along the lines of the Election Review Committee that could launch in August and watch events this year, then decide what changes, if any, should be put in place for next year.