Problems with the 2010 State of the Blogosphere Survey

Technorati have got a 2010 State of the Blogosphere Survey

The good news

It's great that somebody's doing a survey about what bloggers are up to.  I've always found their previous surveys to be really interesting and very illuminating as to what is happening. 


I also don't recall ever having an issue with any of their previous surveys.

The bad news

This particular survey is not as well designed as it could be - for the purposes of this particular survey.  I think it's likely to cause some problems both in terms of responding and the analysis of responses.  In particular the software they are using is not up to standard for a survey of this importance.

Here's what I observed

Format / Process

• There's no 'back' button.  This means once you've answered if you reflect on what you've said just after you pressed the 'continue' button and want to change your answer you can't.  I don't think I've answered an online survey for quite a while which doesn't allow you to go back and change an answer so this must be down to the survey software they've chosen to use.
• You can't change text once it's been written without backspacing.
• You can't move text around in the text boxes provided for your response which I have never ever encountered before with a survey.  I often start jotting points down and then move them around so they make more sense to me and the reader.  That enables me to check the response I've given - but not with this survey.   
• It also has a character limit where it asks for feedback but forgets to tell you this until you've written and posted your feedback.  They got the short shrift version after I had tried several times to remove what I'd originally written with very limited succes for far too long.

The survey questions

• It includes questions which are odd and have obviously not been 'test-marketed'

• Is this your first blog - Yes but I've created others as well - but this is not available as an answer.  Should I do a separate survey for each of my four blogs? ;)
• How many blogs do you currently have?  - Well that depends on whether you're talking about just my blogs or the group blogs I contribute to.  There's no recognition of group blogs which I would observe is one of the features of blogging which has grown over time.

How often do you usually post to your art blog?

• Some of the options are very odd and will distort the analysis of results

• Frequency:  I blog almost daily on Making A Mark.  However there is no option for a frequency of blogging  inbetween "once or twice a day" (ie a response which includes difference of 100% in the frequency of your blogging depending on whether it's once or twice a day!) and "two-three times a week".  In my view there SHOULD have been an option for "more than once a day" "usually daily" and (say) "mostly daily/c. 5 times a week".  I know lots of bloggers who post very frequently but not daily - they will not be identified at all by this survey.  As a result whatever conclusion they arrive at about frequency is going to be distorted.  Above you can see the responses I've got to date on my poll on Blogging for Artists
• Resources created for posts:  a two option 'black and white' answer which does not allow for shades of grey.  So the fact that I post my own photos and scan my own artwork but also use other people's images with their permission cannot be reflected in the answer.
• Duplication:  I lost count of the number of times I repeated the fact that I use Twitter!  This gave all the signs of a survey to which people had added questions at the last minute rather than one which had incorporated validation questions to check whether your first response is consistent.
• It's overly long.  If they'd eliminated duplication it could have been a lot shorter.

It's all silly stuff - but it's frustrating and I think a number of people may well give up before they get to the end.

There's also a bit of a sense that they've decided what the issues are and are now asking the questions to provide the evidence.  Now while I'm all in favour of hypothesis testing this survey felt it closed off to many options too soon to me. 

If you're wondering why I'm writing at such length - it's because Technorati will probably pick up the title of this blog post on Google Alert and hopefully will reflect on what it says and

• remedy the problems before next year's survey. 
• reflect on the issues it raises in relation to the analysis it produces

There again I might just go and link to it on their blog!

PS I'm not here - this posted automatically.  I'm actually getting dressed for a family wedding and will shortly be on my way to a castle in Wales!  Remember - there's no "who's made a mark this week" tomorrow - but there is a post!