Following up on my notes about the 2016 SQLSaturday Roundtable I wanted to write more about the ROI of SQLSaturday. I suppose there are many ways to measure it, but I think it boils down to two views:
- How many people did we train/hours we deliver to our members? That’s what we’re in business to do, right?
- How much revenue did we drive back to PASS, primarily in the form of Summit registrations?
PASS puts a decent amount of money into SQLSaturday. A full time evangelist plus their travel, hosting the site, making improvements to the website, creating the event logo images, responding to web site problems, organizing the Roundtable and putting some direct money (though less now) into each event in the form of sponsorship. I’m not sure of the exact number, but it’s probably $200k-$400k per year, out of a $5-8mil/year budget. In return PASS gets brand exposure, a continuing list of new attendees and CRM type data on those returning, and a chance to further serve those. They are a super-sponsor, getting the entire list, an advantage no other sponsor gets. Good deal? Maybe.
In a perfect, or even better world, we’ve see a nice upsell path from attending SQLSaturday to attending the Summit (the money maker). In practice that number is very low, at least as far we can tell looking back five years and not having a perfect “member id” for each member (they often use different emails for SQLSat vs paid events). As is currently assessed SQLSaturday is “losing money” – which is acceptable or not depending on the state of the budget, whether you think “losing” money on anything is bad, and how you see the overall role of PASS. That lack of growth has contributed to a view from at least some Board members that SQLSaturday is decreasing Summit attendance/growth (though I hear it grew 10% in 2016), a position so far not supported with data and one that seems non-intuitive to me. Those two things plus some short of budget shortfall due to an overly ambitious revenue projection have led to cuts in how much money goes to events from PASS for sponsorship.
Why don’t our attendees go to the Summit? Easily the number one reason is cost! It’s a very large spend, many are reluctant to even ask, and fewer still get approved. It’s often a multi year effort, or one that is done when negotiating initial salary and benefits. We can’t (probably) make it cheaper long term, but we would surely work on giving those not-attended-yet people more of an incentive (first time cost of $1000 for example). We could do more to reach the decision maker directly. There’s no doubt I take this “failure” to drive Summit sales personally. Those of running these events are pro-PASS, pro-Summit, and we do what we can, but we’re running the event. Our job is to show people the value of learning, get them thinking about learning more, and to get them into the system. The job of PASS and its marketing team is to find ways to reach those contacts and guide/incent them to attend other events, paid or unpaid. I’m glad to do what I can to help, but if this isn’t the target audience to market to, what is? How do we not see this as a marketing failure? Setting up a table and staffing it with a regional mentor is a start, but it’s superficial.
I was stunned to hear a Board member tell me people are opting out of Summit because they can just go to more SQLSaturdays (leading him to think we should do fewer of them). Is this true, where is the data? It defies intuition at best. No one skips the Superbowl to go to 4 local high school games, regardless of the quality of those teams. If it it true, then what? That’s an interesting discussion. They go to multiple events because they enjoy them and can afford them. How many go to multiple events per year? I’d be surprised if that is a large number, not counting speakers.
I heard about “speaker exhaustion”, a problem that I don’t see but might exist, and is easily fixable. I heard about reduced sponsor participation, or at least the inability to scale it up as we grow. Both in the context of reasons to do fewer events and to make them smaller.
I get PASS has to pay the bills. I’m not opposed to being smarter about how we send out money. For example, lets offer more money to first time events, or to events that cannot find a free venue. I’m not opposed to doing less for a year if we had a shortfall. Maybe it’s time to wean from that entirely, but that cost – about $60k – is noise in the PASS budget.
Ranting and frustration aside, it’s the nature of a quasi-not-for-profit to struggle with balancing the need to raise funds vs using those funds to do go good. What I don’t want to see happen is a return to the bad old days where PASS spent basically nothing on anything besides Summit, a game of make money so we can grow staff and make more money, over and over. That’s a tough balance without some guiding principle or goal that I’m not sure we have. I’ve wondered more than once if we didn’t need PASS.COM and PASS.ORG as two different companies, one for profit with a mandate to put x dollars/percent into the other.
Why does this all matter? I feel like the Board is making decisions about the growth, size, etc, of SQLSaturday based on an incomplete view of the world. We can cap spending, cap the number of events, do other defensive things, or we can ask “how do we change the game so we can train more people?”. Steve isn’t wrong to think about 500 events a year. Not events for the sake of events but to reach people that want to learn, grow, join the craft, be part of our community. That’s why we started this, why so many of you lead and speak and volunteer at these events. If we can help with the ROI, that’s good. If we have to get less support, let’s be smart about what support matters. We need a longer and better conversation about the two conflicting but valid views of ROI. I think we can achieve both, but we don’t do it by pulling back.
SQLAndy 
Andy,
Those are some interesting thoughts.
Just FYI, if it were not for SQL Saturday (and meeting you and your peers) I probably would have never even known about Summit.
Another approach to greater ‘Market Penetration’ might also be the session recordings. They could be seen as a discount alternative to flying coast-to-coast. (Although I would like to buy them, $995 is still too rich for me.)
Thanks.
LikeLike
$995 is a lot. Tough to lower cost too much without impacting Summit attendance, but we could be smarter – sell 10 packs for $250, get it down to something the boss can put on a credit card. We’ve also talked a few times about sharing content from previous years. Hoping we see some changes on all of that this next year.
LikeLike
I think the PASS Summit is great but it is based on a somewhat dated model. Asia is a massive untapped market that is totally ignored by pass.org. As an exclusively USA organisation, pass.org cannot see across the Pacific and the huge cash cow that is awaiting. It is quite possible that an Asian based organisation imitates the PASS Summit and steals the Asian fan base away from pass.org.
The cost of the PASS Summit is not at the correct price point. The cost of attending the event for a resident from Seattle is X + bus fare. Indians, Chinese, Australians, and New Zealanders pay X + $USD 1,700 airfare + USD 750 (5 nights accomodation) + 2 pre-con days (to make it the trip worthwhile). The long haul foreigners do not receive a discount and they do not receive any special treatment. There isn’t even a map given the foreigners.
Pass.org is becoming very lazy.
LikeLike
Bouncy, I think PASS has taken a step or two towards being more international over the past 5 years. Enough? Probably not. We used to have a Summit in Europe but it never quite worked. I’d love to see a Summit, SQLBits, Super SQLSaturday in those far off places. It’s a big bet for PASS to organize one, both because of the cost model they use and because it would be much more remote organizing then in the US one. What I hope we see is chapters and events grow to the point that someone proposes a regional event – a SQLBits/Super SQLSaturday type thing at a decent price point, and then I think PASS would help but probably not own the event (my guess).
The note about the map is interesting, I’ll forward that along.
Lazy? Maybe. I think its more complex than that. I do see that PASS could do more to incent and demonstrate international growth.
LikeLike
Late to this conversation, but do SQL Saturdays have to be free, and how much would reduced attendance at the Summit be offset by charging say $25 or even $50 to attendees at SQL Saturdays (in addition to the lunch fee)? We have one SQL Saturday per year in my city, so I would be willing to pay $100, and it would be a fraction of the cost of Summit proper, not even considering travel expenses. Granted, its one day and perhaps not as much content, but a lot of that I’m probably not going to be able to use for years. And, I’m networking with more people whom I am more likely to interact with professionally, they being from my local area. At $100, I’d still be willing to drive to a couple other SQL Saturdays as well. Just a thought.
LikeLike
Steve, I love that you’re getting enough value to pay, but paying changes everything. We get our venue for free because we don’t charge for attendance. We get speakers for free because we don’t charge for attendance. But assuming we could address those, the biggest concern remains what of the impact on attendance? We see 10-20 people pay $129 for a one day class here in Orlando each year, then we get 400 show up on Saturday for free. Is free the difference? Or because its Saturday? Or the different format?
LikeLike
I go to quite a few SQL Saturdays, (was just at one last weekend, will be at another in 2 weeks), and after reading your post a very simple thought came to me — while PASS usually has a vendor booth at the SQL Saturdays, the presenters are not encouraged to talk about or explain to the attendees what PASS actually is and that there is a big event called Summit that happens once a year that is the Biggest, Baddest SQL Training event in the world. An idea is to provide speakers with a different / changing set of 2 slides to start off their presentation (so every presentation at the SQL Saturday doesn’t have the same slides), showing pictures of Summit, explaining what and when Summit is, what PASS is, etc. How to join and why…
Because I know for a fact that many people just go to each vendor booth and drop their ticket into the raffle box and walk away as fast as they can hoping to not get pulled into a conversation. How many people are actually stopping long enough at the PASS booth to “get it”? It is very likely that many of those attending SQL Saturday who are new don’t leave the event “getting” what PASS even is or that there is a Summit event. THAT is something we can easily change, and the speakers would be enthusiastic to help spread the word if it just took 1 or 2 slides and was quick.
LikeLike
Mindy, you’re right, it’s easy to do and repeating it 5/6 times during the day would at least guarantee they had heard of it! I think one of the weaknesses we have is that we don’t have an ineffective way to drive that kind of change into the formula.
I think its time to revisit sponsor engagement overall. Maybe what we do is the best that can be done in the format of our events, but we should look.
LikeLike
I have been involved in SQLSaturdays locally in Albuquerque and Colorado Springs. I have been a presenter both at our local monthly lunch and several SQLSaturdays. I am looking forward to the Denver February 2017 SQL Saturday.
I would love to attend Summit but it is cost prohibitive. I can not afford to pay for it myself and I work for government where it is very difficult to get approval for training and travel.
I would be very disappointed if SQLSaturdays were reduced or funded less fully.
LikeLike
Leslie, thanks for commenting. It is expensive and I doubt we can change that, the logistics are tough. One approach is to try to earn your way in as a speaker. Tough, but not impossible. The other is if(!) you change jobs, consider negotiating annual or bi-annual attendance as part of the comp.
For most events not getting the $500 from PASS won’t be a deal breaker. Where the funding matters more is in the salary/travel for the evangelist that coaches/grows new events and supports existing ones, and the IT spend (hosting, improvements, Guidebook, etc). I think it all deserves a better conversation with the members than PASS has done so far.
LikeLike
I found out about SQL Saturday’s 2 years ago and registered using my personal email. After attending 2 Saturdays I lobbied hard to go to Summit. Since work paid for Summit, I used work’s email account to register. While its a single data point, I am willing to bet you lose many direct conversions in this fashion.
LikeLike
Bob, I don’t know. Supposedly they put a lot of work into matching them up, but I’m sure there are some gaps and the gaps maybe add up? I don’t know if they can make the list public due to privacy, but they could at least report on it year over year. I can think of a few questions:
– how many Summit attendees attended a SQLSaturday in the past year?
– how many Summit attendees attended one or more SQLSaturdays before attending the Summit?
– how many attendees are affiliated with each chapter? (Big cities would do more, but we could measure year over year to see if they are helping)
LikeLike
“The job of PASS and its marketing team is to find ways to reach those contacts and guide/incent them to attend other events, paid or unpaid. ”
Ahh, I think the “Job of Pass” may need a little work. Is the purpose really to get me to spend more money?
LikeLike
Hey Ray,
Thats my wording, not theirs, but I did say “paid or unpaid”. The tagline is ‘connect, learn, share’. As a career/craft based professional org, if someone comes to one of our events isn’t in the right thing to continue to communicate to that person there are other learning opportunities that might interest them, free or paid? (Preferably with equal weight on free and paid!)
But you do point out the problem, PASS is far more interested in marketing paid events than free ones. It’s made more complicated because a lot of the free events are local and micro-scale, the local groups “own” the marketing, compared with the house events (Summit, 24HOP, BADays now). In the end the Summit pays the bills for any/all give back to the free events. I think its important to make sure SQLSaturday attendees know about the Summit, but equally or more important to just engage them year around from a national/global view and not just the local one.
LikeLike
As one of the people who attends SQLSaturdays and not Summit, I can confirm that I’m more likely to attend 100 SQLSaturdays than Summit – but that’s entirely due to my job.
1) I can’t get the whole week off to go to Summit no matter how much I want to and 2) It’d be entirely on my own dime, my employer won’t even pay for SQLSaturdays, much less travel + lodging + Summit ticket. Even trying to get recordings would be difficult (but I’m almost ready to fight that fight).
I’m sorry for being part of the problem, but my hands are kind of tied at this point, and I’m sure I’m not the only one in this boat. Maybe like you said, the hurdle is in helping convince employers that it’s worthwhile to go to Summit (my employer totally COULD afford to send me, they just don’t). I know I’d appreciate some help in convincing them.
LikeLike
Fred, I don’t think you’re part of the problem. I hate to think we frame it that way at all. PASS should be providing opportunities to learn and connect, some paid and some free, and some will fit you and some won’t. A win from my perspective is if we’re finding a way to serve you at some level.
I think your comment about needing some help to convince them is fair – just hard. I know one thing that has worked for me during negotiations is to “this is important to me”. Rather than arguing about how much great stuff I’ll learn or all the contacts I’ll make, I present it as part of my identity. I’m a SQL guy, this is where SQL guys go, and I want to be able to continue that if I join your team. Essentially its an emotional argument:-) If they are ready to hire you another $5k a year isn’t a deal breaker. Where we get stuck is when we have to make the case for it year over year – that’s mostly a losing game.
LikeLike
Great article. Just a note on the real world effect that PASS can have. In 2012 I decided to make a career change; from that of a business/financial analyst to an aspiring DW/OLAP developer – I discovered PASS on my own which was pivotal in this transition – it served as a conduit to the SQL Server world and most importantly was a primer on where to start. I was hungry so I chose to invest part of my savings into PASS Summit 2012, although I felt like the most novice, inexperienced person in every session it was an amazing, nearly life changing experience for me. I sat through a career focused session Andy Warren’s led , I remember that clearly. I also gained virtual mentors from that experience and PASS in general. Since then I’ve attended 3 SQL Saturdays, but Ive also been wanting to attending PASS Summit again. Live has changed considerable and the Summit’s expense doesn’t fit into the family budget. Ive had 2 employers since and no one has been remotely willing to sponsor me for the Summit – there may also be a cultural variable here; in my professional experience Ive not yet met a db professional thats actively aware of PASS so it may come off as esoteric (Outside of SF and SD here in CA, I really feel a concentration of the community resides in the East and Pacific NW). My point – I don’t feel one can compare Summit to the Saturdays. Sponsorship may help especially for newbies, there was no distinguishing difference (cost/administratively) back in 2012 between me and a mid-senior level attendees; I felt lost, lonely and confused, but the drive and intrigue brought eventual normalization. I really hope Summit becomes more attainable, I know there was the pre Dec 31st discounted cost of $900 which is a great idea. PASS and all of you are amazing, I really cant express the importance of you as “virtual mentors” for people like me.
V – Santa Barbara, CA
LikeLike
Vinnie, I think we still have work to do on the first time experience – you’re far from the only person to feel that way about attending, it can be overwhelming (especially to introverts!). The early bird pricing is a great deal but I find few companies willing to do it, they prefer just in time spending. Thanks for commenting and hope to see you again at an event!
LikeLike
I’ve worked with SQL professionally for 18 years in the UK. I’ve been going to usergroups long before they became PASS chapters. I also attend a few SQLSaturdays and the yearly SQLBits. I’m aware of PASS from the branding of SQLSaturdays, Summit marketing (and laptop stickers) but I confess I have no idea how much PASS subsidises these events.
As someone who has never been to Summit, I’ll just offer up my reasons here. Please don’t take offense, it’s just as I see things.
Training –
In terms of learning there are a lot of online courses and training providers that you can view content again and again. You can chose targeted training for what you need now, whereas at a conference you might attend something interesting and never get the chance to use the knowledge / forget it completely.
Time –
Dedicating a specific week to travelling & training is difficult to plan months ahead (when the tickets are more affordable)
Location –
There are user groups and events all over the Uk, Europe , infact half the world that are a lot more accessable to me.
Quality –
I totally disagree with the superbowl analogy. The biggest is not always the best. The big name speakers can all be seen elsewhere throughout the year.
Cost –
Some events are free, others cost. Online training is cheap and even those costs can be paid via subscription, spreading them out. They are all a fraction of conference costs.
A lot of us don’t have generous training budgets that would allow us to attend paid conferences, let alone travel internationally to Summit. If your ROI is purely PASS attendances then you should contentrate your efforts closer to home.
Attending PASS Summit is just a pipedream to the majority. Those that are lucky enough may have to sign contracts that tie them to their employers or pay back the costs of flights, hotels as well as the summit.
I do appreciate the community, I do appreciate that attending these events is about the experience but those are difficult things to sell to your boss to sign the cheque!
LikeLike
Dave, fair points, though we could argue some about the Superbowl part. You can see a few of them at a given SQLSaturday, but not all. There’s also the impact that just having a top-of-the-pyramid event has on driving competition to be better speakers; smarter, better presentations, interesting topics.
I will come back and read your comments again in a bit, lots of good stuff there.
LikeLike
My attended my first pass summit in 2016 and I have to say I learned more at SQL Saturday than I do at pass summit. Smaller group presentations at SQL Saturday make it more personal and very direct. The summit has large number of attendees, which slows down the presenter with a lot more questions. And of course for the price + travel + hotel. Well over 2 grand for only 3 days not sure if it was worth it.
LikeLike
Kenny, thanks for commenting. Its a different vibe with crowds. Not worse, just different. It’s hard to keep the price low once the event gets large, hotels and convention centers force you into a lot of things, such as using their food. It’s not uncommon to be charged $40/gallon for coffee!
LikeLike
I’d love to attend a summit but I doubt that’ll ever be in the cards for me with the cost and time/distance involved. I have been to a SQLSaturday but there’s only be one in my area (Toronto) in the last decade as far as I’ve heard. PASS, as Bouncy alludes, is almost all about the U.S. whether we’re talking summit or SQLSaturday.
LikeLike
Interesting thoughts as usual Andy. I do find it interesting that PASS Summit attendance (at least as announced at each Summit I’ve attended) has grown since SQLSaturday started, but now is it is being said that it is a hindrance to Summit growth? I don’t get it.
PASS can control the growth of the SQLSaturday brand, but they can’t control the growth of local training events. SQLBits, SQL Relay, SQL Konferencz are all very successful events without being under the PASS umbrella.
LikeLike
Jack, you’re right about local events. I hope that PASS grows and supports them too (if the organizers want/need support). Sponsorship is one way of doing it, there could be others. But more to your point, we’ve spent a lot of years teaching people to fish. Once you know building a local event is possible and you’ve done it, doing it yourself is less daunting. I hope that we won’t see a good pattern splinter due to mismanagement from PASS, but in the end what matters is that we train people and connect them.
LikeLike
Forgive the bit of an unorganized rant here:I always though the purpose of PASS was to spread knowledge about SQL Server and its usage though the community. I always saw SQL Saturdays as one component of that and not a commercial for Summit. The fact there is so much free knowledge and local user groups and SQL Saturdays are the reason I joined PASS.
I am not a full time DBA my company does not see the need for a full time DBA on staff and as such I know they don’t see the value in sending me to Summit. I have not been to Summit since I would have to pay the full price and accommodations out of my own meager budget.
I have been to Summit-like conferences for other vendor’s technology and to tell the truth it ended up feeling more like a commercial for the company’s new releases and products than anything. Sure there were networking events you could go to in the evenings. Sure there was training available if you wanted to pay extra to be able to go to the training, but I knew my attendance was barely approved and any expenses beyond that were going to be questionable at best. I actually in the end felt like my company paid thousands of dollars to send me to a commercial for that vendor’s product. I almost felt like it was the type of thing the vendor should actually have paid me as a customer to attend and not the other way around.
I am reasonably happy at my current job and not looking to go network with professionals from around the country for new opportunities, so I have always had difficulty seeing the value in Summit. Especially since being a DBA is really just one of many functions within my job.
I have been a part of other user groups and they are usually sponsored by the vendor and end up just being like that summit I attended a way to distribute more marketing not a way to build up educate your user community. I do not mind the occasional sponsored mailings from PASS and always stick around to hear the sponsor info at the end of SQL Saturday (I can say from the past not a lot of people do and most only stick around for the giveaways at the end). At least with PASS it is sponsored by many vendors so it does not become the mouthpiece for one, and that is one of the things I really appreciate about it. You do get that knowledge transfer and peers that you can compare notes with without the overwhelming push from just one vendor.
LikeLike