[Foundation-l] Regarding Berkman/Sciences Po study

Sat Dec 10 10:52:57 UTC 2011

(I originally sent this to WikiEn-l - but intended to send it here to
foundation-l... my email reader got confused..)

Hi Dario,

This proposal went through a long review process, involving community
> forums, the Research Committee and various WMF departments since early 2010.
>
> The Berkman research team first approached WMF to discuss this study in
> January 2010. They suggested a protocol to recruit English Wikipedia
> contributors to participate in an early version of this study by March 2010
> and posted a proposal to the Administrators’ noticeboard to get community
> feedback [6]. The community response at that time opposed the proposed
> recruitment protocol (posting individual invitation messages on user talk
> pages). It was suggested instead that the recruitment should be handled
> through a CentralNotice banner to be displayed to registered editors, but
> concerns were raised on how to minimize the disruption.
>

This is not a good summary of the conclusions there at all; and it is
worrying that it has been read that way...

You seem to have taken that discussion as implicit approval to run a
CentralNotice banner - although that was certainly suggested as an option
at the time I think it was reasonably expected for further community input
later down the road. Certainly when I supported the suggestion of some sort
of targeted site notice I envisioned a text link, or something.

> Throughout the review process of this recruitment protocol, the research
> team received constant feedback from the Foundation’s legal team, the
> community department, the tech department and the communication team before
> the campaign went live.
>

But not the community?

> The campaign was announced in the CentralNotice calendar one month before
> its launch [11] and the launch was with a post on the Foundation’s blog.
> The banner was enabled on December 8 at 11:00pm UTC. 800+ participants
> completed the study within a few hours since its launch. The banner was
> then taken down by a meta-admin a few hours after the launch due to the
> concerns described above.
>

Again; not announced to the community. There was a clear an present
communication failure here.

> We realize that despite an extensive review, the launch of this project
> was not fully advertised on community forums. We plan to shortly resume the
> campaign (for the time needed by the researchers to complete their
> responses) after a full redesign of the recruitment protocol in order to
> address the concerns raised by many of you over the last 24 hours. Here’s
> what we are doing:
>
> • Provide you with better information about the project
> We asked the research team to promptly set up a FAQ section on the project
> page on Meta [13], and to be available to address any concern about the
> study on the discussion page of this project. The project page on Meta will
> be linked  from the recruitment banner itself.
>
> • Redesign the banner
> We understand that the banner design has been interpreted by some as
> ad-like (even if the goal was to make clear that this study was not being
> run by WMF, as it implied a redirection to a third party website for
> performing the experiment). In coordination with the research team, we will
> come up with a banner design that will be more in line with the concerns
> expressed by the community (for instance by removing the logos from the
> banner).
>
> • Make privacy terms as transparent as possible
> Upon clicking on the banner, participants accept to share their username,
> edit count and user privileges with the research team. The previous version
> didn’t make it explicit and we are working to address this problem. To make
> the process totally transparent we will make the acceptance of these terms
> explicit in the banner itself.
>
> Once redirected to the landing page, participants will have to accept the
> terms of participation in order to enter the study. The project is funded
> by the European Research Council: the data collected in this study is
> subject to strict European privacy protocols. The research team will use
> this data for research purposes only. The research team is not exposed to
> and does not record participants’ IP addresses.
>

You need to tell this *to the community*. Otherwise the discussion will
simply strike up again once you re-enable it. I notice you posted this
exact same message to wikipedia-en-l. The lack of recent discussion on that
list should tell you how effective that is as a communication tool.

The vast majority of English Wikipedia discussion occurs on-wiki, and the
vast majority of editors prefer discussions to occur on-wiki. If you want
to interact with the community, and in this case I think you have to, then
you really have to do so on-wiki :)

We would like to hear from you on the redesign of the banner to make sure
> it meets the expectations of the community and doesn’t lend itself to any
> kind of confusion. We will post the new banners to Meta and try to address
> all pending questions before we resume the campaign.
>

Most en.wiki editors don't hang out on Meta - and I think it is reasonable
not to expect them to. Especially as this is purely a project focused on
English Wikipedia; it needs to be discussed on English Wikipedia. If you
need help with the right places/protocol then I would be happy to oblige.

This once again highlights the huge disconnect between Wikipedians, the
foundations and the various higher level commitees. It's the same every
time - something big appears, the community get cross/upset/confused, the
foundation etc. express incredulity... and no one talks.

My final comment is this; I am fairly active around here. I am on
foundation-l, I read meta for important notices, and I am active on
en.wiki. And the first time I knew about this (since the Admin Noticeboard
discussion some months back) was when the banner appeared. So how anyone
else stood a chance of giving input I do not know :) Whilst much of the
discussion was public, it was "hidden". And the key failing here is in not
making it open and accessible.

Tom