Hi WSC,
thanks for the detailed feedback. Here's some background on what guided the design of
the survey:
Negatively phrased questions
All the questions were compiled based on actual input we collected during the pilot. As
outrageous as it may sound, editing Wikipedia as a potential threat to one's career is
an actual reason some participants reported and we want to assess its importance as a
potential barrier. If you google "tenure blogging" you will get an idea of how
hot the debate has been around the question whether blogging is a risk or not for
one's career (especially for non-tenured faculty). Something similar may happen (or
may have happened in the past) for Wikipedia and we want to capture this i the survey.
Starting the section with...
The order of the questions in each block is randomised for each participant, so it was
just bad luck :)
Individual motivations vs shared perception
This is the most delicate issue of the survey, for which we considered several alternate
designs. We want to contrast the participants' perception of Wikipedia participation
in general with one's individual motivation to contribute or not to contribute. The
rationale for this is that we noticed that participants often can dissociate their
judgment on statements they would endorse as members of a professional category
("editing Wikipedia does not count towards improving one's CV") from
judgments on what represents an individual barrier to participation ("even if editing
Wikipedia does not improve my CV, this is not a reason for me not to contribute").
Phrasing questions as regarding one's peers is a way to have a participant focus on
shared perception as opposed to individual motivation. The reason why we put general
questions upfront and individual-motivation questions at the end is that we believe the
noise added by framing questions "from general to individual" is less important
(for what we want to study) than if we ordered them "from individual to
general". To address this properly we should randomise the order of the two blocks,
not just the order of questions within each block, that's something I can look into.
Dario
On 7 Feb 2011, at 19:00, WereSpielChequers wrote:
I appreciate that phrasing questions neutrally can be
somewhat
difficult. But I think there are too many leading or negatively
phrased questions to produce useful information. Starting the section
with "Editing Wikipedia may damage one's scientific reputation" rather
sets an anti wikipedia tone.
Also I'd switch the sequence between individual and collective
perception. Putting the section about the individual before the
collective section would start with something that respondents should
more easily be able to answer - some people simply won't feel that
they can answer questions on behalf of people in their field if they
are unaware of those other people's views.
WereSpielChequers
On 7 February 2011 12:16, Dario Taraborelli <dtaraborelli(a)wikimedia.org> wrote:
All,
the Expert Barrier survey is ready to run. We took into account most of the suggestions
we received during the pilot phase and we are planning to release the survey by Wednesday
morning London time, in time for the Imperial College Recruitment Drive [1]. Here is a
link to a live preview:
http://nitens.org/ls/?sid=21693
(some blocks of questions are hidden depending on how participants answer in the first
screen when asked whether they ever contributed to WP)
If you have a moment to give it a try, comments are welcome by Tuesday night PST.
Best,
Dario, Giota and Daniel
[1]
http://en.wikipedia.org/wiki/Wikipedia:CONTRIB/Imperial
_______________________________________________
RCom-l mailing list
RCom-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l
_______________________________________________
RCom-l mailing list
RCom-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l