I’ve been kicking around several ideas in the program committee and a couple of them have to do with what information PASS releases, specifically information about the speakers. The general question I’ve been trying to come up with ideas about, and the subject of this blog post is:
Should PASS Release Speaker Evaluation Scores to the public
As with all things, there are goods and bad’s to releasing this data. And there are even more possible ways to release the data.
Personally, I’d like to release aggregate scores for every session at every summit. I’m not sure that is a practical option though. I’m hoping some people out in the community might tell me that I’m thinking this through too much and no one cares, after all many of our speakers use speakerrate.com which is totally open.
Everyone who purchased the Summit DVD’s would know when looking at their session lists which to focus on first
Every speaker would know exactly how they compared to others, I think this would be especially helpful to speakers who are just starting their craft.
New conference attendee’s would know which speakers have better ratings and could plan their itineraries appropriately.
Every speaker may not appreciate their scores being published (especially lower rated speakers)
Privacy: See above
We could simply require that if you want to speak at the Summit, we will release the results. We could put some language in the terms that are agreed to by the speakers, actually I think we could do this without changing the current terms but, I do not think this is the right thing to do, it just doesn’t pass the smell test for me.
We could include an opt-in on the speaker terms, which would allow disclosure. While this would be the easiest option to implement with the least amount of backlash, I don’t think it really accomplishes much, mainly because without the complete picture of the scores you wouldn’t know how the speakers ranked or if you were a speaker it would make it tough to know for sure where your session stood.
We could release the complete score list to every speaker, this would solve the speakers knowing where they rate in the crowd. Unfortunately It wouldn’t help those in the community know which speakers were the best.
As with all things PASS I’m open to suggestions, and maybe I’m missing something, let me know
Last time I wrote about limiting speakers to 1 session per summit, I appreciate the feedback, and thought id take another turn at this topic.
As I was reading the comments it reminded me that last year, I was asked by PASS marketing to pull a list of what we considered “new presenters”. I thought I would revisit that list and give some of the statistics I discovered.
I want to start by saying I did this research almost 9 months ago, and I didnt reverify that this information was correct enough to post, it was pulled from their submitted bio’s. So feel free to point out if something is wrong.
We had 85 unique community speakers in 2009. Of these 85 I was able to identify 17 people who were new to the summit within the prior year (08 and 09)
Gail Shaw 08
Grant Fritchey 08
Denise McInerney 08
Denny Cherry 09
Jacob Sebastian 09
Brent Ozar 08
Rod Colledge 09
Jamon Bowen 09
Jonathan Kehayias 09
Michelle Ufford 09
Ravindra Gurram 09
Jason Massie 09
Rob Garrison 09
Glenn Berry 09
Kendal Van Dyke 09
Kevin Guinn 09
Trevor Barkhouse 09
Every speaker on this list wasn’t new to speaking, they were new to speaking at the PASS Summit. Which is exactly the kind of new speakers we’re looking for.
At first glance Id be willing to say that PASS is already doing a pretty good job of involving new speakers. Using these numbers it looks like we already incorporated approximately 20% new speakers within a particular selection year. While ultimately I think that number could(should) inch up even higher, Im happy to know that we’re not doing so badly at this that I want to go visit an Access conference just to raise my spirits.
Following up to Andy’s series about growing the pool of speakers, I thought I would detail an idea we’ve been kicking around for this years summit.
First some history
In 2009 PASS accepted 585 community abstracts submitted for a total of 113 community sessions slots that were available (including 10 pre/post conference sessions) of those 80 were regular sessions, and 23 were spotlight sessions. We had 30 speakers give 2 sessions including the 10 pre/post conference sessions. We normally ask that speakers accepted for a pre/post conference session also present a spotlight session, so that every attendee of the conference gets access to these high caliber speakers. This left us with 20 regular speakers presenting 2 sessions in 2009.
The big idea
In order to give more speakers a chance to present at the annual summit, were proposing limiting all community speakers to 1 primary session per summit. There would obviously have to be exceptions for panel sessions and co presenters since we wouldn’t want to discourage those types of sessions. The benefits as I see them are that we’d open up 20 more slots, give or take from year to year, to new speakers thus allowing others in the community an opportunity to present. The downsides (or risks) as I see them: We stand to potentially loose coverage if we receive no abstracts on a particular subject that would be a currently chosen speakers second session. Cost, that is 20 extra comped admissions to the summit. Pass would need to decide if the value of these extra comped admissions are worth the expense. What I mean to say is if we spend money on those extra comps, that money couldn’t be used on some other priority.
I’d also like to mention that in my years on the program committee, no discussion of comps has ever occurred while deciding to choose one speaker over another. This however is different since it would be a policy.
So, what say you? keep the current limit of 2 per speaker, restrict it to one, make the limit 15?
Here comes High Gear
We received 53 applications this year from members hoping to help out on the program committee. That’s a huge number for us, When I joined in 2007 I think I was accepted by default because we didn’t have enough volunteers. in 2008 there were approximately 20, last year we had 23. Obviously we’re gaining some attention in the process of making the education program at the annual pass summit the best year after year.
Selecting the abstract review teams
With so many applicants, selecting the teams was quite a chore this year. Its like interviewing for an open position at your company but instead of having 1 position, you have 17 open and instead of a hand-full of qualified applicants, you get a boatload. That pretty much sums up the experience every year but, this year it was twice as bad as I remember it being in the past (possibly because of the doubling in applicants). In the end, the quest to fill these teams took quite a bit of time, unfortunately just like selecting employees for your day job, there is really no best way to make the selections. I should have probably just used a dartboard for the selections but, I didn’t want to disservice those that had taken the time to apply. After quite a few iterations of making sure everyone was in the team that best leverages their skill sets, the new volunteers were notified.
The program Committee Portfolio changed hands in the PASS BOD this year, the new owner is Jeremiah Peschka. Overall my experience with Jeremiah has been the same as my experience with the prior 2 Board members I worked with on Program, they’ve all been nothing but great. We have already started batting around a couple of major changes to the Program Committee.
I tricked convinced Lori Edwardsto join the program committee as an adhoc project manager unfortunately we haven’t found a jazzy title that sticks yet, so she’s currently known as the task team leader, I hope we can properly anoint her later. With 17 applicants added to the abstract review teams, we moved the other 36 people who applied into a task group that Lori has the joy task of managing. Having a dedicated volunteer leader to manage the side projects that always come up should change the way we get things done and hopefully without me being the bottleneck, things will happen even faster. This year the program committee has many things we want to accomplish, in addition to the regular things that we have to do to put on the summit every year. I am hopeful this setup will prove to be one that works well, and we can refine and document the process so it can be reused in future years.
Abstract Committee Abstention. In the past we didn’t have a policy about the abstract review volunteers submitting abstracts in the same track as they are reviewing abstracts in. This was usually handled internally to each track and that member abstained from all discussion and ranking of their abstract. This year however, we alerted every potential abstract reviewer that they wouldn’t be allowed to submit abstracts to their review track. This rule may exclude some volunteers from reviewing abstracts in the future but, it just made sense from the transparency standpoint.
New Summit Management Software
The new summit software selection is causing some of our critical timeline dates to need to be adjusted. I’m not in a full panic defcon 5 mode yet but, If we don’t have a working environment in the new software by the end of next week, Ill be pulling the panic alarm. Configuring a site to accept abstracts, and manage the speaker experience takes an unbelievable amount of time and right now we’re still waiting for the final paperwork & approvals. Sliding the call for abstracts back doesn’t effect much of the planning for the summit except it compresses our timelines for getting the sessions selected and posted to the summitt website. Currently we’re aiming for a mid April call for speakers opening, with the actual call running approximately 30 days.
Brad started working on this years update/changes to the speaker resource pages on the sqlpass website. These sorts of changes are always interesting since they cross over a couple of portfolio’s. I’m hoping that we can collapse the 2 pages currently posted into one page that is less confusing, or potentially 2 pages that work better together and don’t overlap as much as the existing pages.
Lots of new ideas coming, expect a new blog post tomorrow outlining one of the more controversial (potentially) Ideas I’d like to get some community feedback on.
Listening to the SQL community over the years, I thought Id take a second to put a quick thought out about why I believe PASS makes almost every decision it makes as an organization.
As a non-profit, Its all about money value, and its use to support the community. Almost every decision that is made within the organization has a value cost, whether that value is actual dollars that need to be spent, or if its a volunteers time, nothing is free or limitless. Just because Microsoft decided that the top tier SQL engine will now cost 58k per processor, doesn’t mean they are giving that money away freely to user group organizations.
If you’re unhappy with where the organization is allocating its resources, Contact them directly, or probably even better contact all of the board of directors (protip: email format ) let them know you want things to have a different priority, if that fails, they have elections every year, VOTE!