I just had the privilege of serving as a juror for the 2014 Canadian Weblog Awards.
I, along with 33 other jurors read through 530 blogs over 26 categories, from activism to agriculture, from health to humour, in a effort to identify some of the best blogs in Canada.
These kinds of things are always very hard to judge. What differentiates “good” from “great”? How do you judge a mother’s blog about raising a special needs daughter against an corporate blog on international markets?
Or even a travel blog by a couple backpacking across Brazil on a shoestring to raise climate change awareness against a commercial travel blog sponsored by outfitters and airlines?
How do you account for personal taste or your own biases?
The organizers of the CWA, Elan and Aidan Morgan have made an admirable run at keeping the judging objective and consistent across jurors.
We judged the blogs on 10 criteria for content and design, awarding 0-2 points for each item. (And by the way, kudos to Elan and Aidan for being transparent about the judging criteria.)
But still, even with metrics, it was hard.
For instance, if a site’s functionality worked perfectly, is that a 1 for “average” or a 2 for “excellent”? If most sites function properly (and they do), then I’d think that’s a 1. But then how would anyone earn a 2? More functions?
However the various jurors chose to interpret the scoring metrics, I think the winners represent a valid cross-section of high quality Canadian blogs. They truly are exemplary.
Sure, there may be quibbles over exactly which ones earned which awards and probably some deserving blogs dropped off the bottom. I don’t know how the aggregate scores looked, but I’m guessing that the difference between placing and not placing was probably a matter of just a few points in the final round.
But when I went through the winners, I saw many that I had ranked high and nothing that I had ranked low. I’d agree they all deserved a spot on the podium. So overall the process worked.
The only suggestion I have for Elan and Aidan is this — have jurors do a first round of judging on 2 criteria that would serve to eliminate blogs from further judging and thus reduce the work for jurors:
- Is the blog Canadian?
- Does the blog fit the category?
As a means to spotlight some fine Canadian blogging across a breadth of genres, I think the Canadian Weblog Awards does an admirable job. Congratulations to the winners and everyone who was nominated.
We actually do manually check each nomination for being Canadian and for appropriateness for category before they even reach the jurors. Even so, 2 non-Canadian blogs slipped through and a handful were later deemed not appropriate to certain nominated categories, but that was over nearly 800 separate nominations, so I’m not concerned. Category Fit is already an explicit part of the process in the second round to catch any that sneak through.
Thanks Elan. You guys really have it together! I picked up the 2 non-Canadian blogs but wasn’t sure if there were others. Also didn’t realize our “Fit” score was a backup to the system already in place. Thanks for clarifying that.
What you’ve accomplished with this award is quite an enormous undertaking. You’ve really got it down to a system. I’m glad to have had a chance to take part.
And thank you again so much. You jurors put in a lot of work, and I’m thrilled that you were there!