Since it does seem very close to going live, could I ask if plans have been made for how to handle announcing the arrival of this feature and any post-implementation problems? Hopefully there won't be any or many, but are there plans ranging from "rollback completely if things go awfully wrong" to "make adjustments as needed and be responsive to concerns raised"? And how much input exactly will ordinary editors have post-implementation? Is the interface flexible and can be changed by editors or admins, and which bits can only be tweaked by developers (either using common sense or following a community poll or Bugzilla request or request somewhere else)? I ask this partly as someone who (with others) may have to deal with any massive disputes or edit wars that break out over this if some aspects of flagged revisions or its interface are editable and changeable on-wiki (presumably in the Mediawiki namespace, editable by admins only). I think the ability for the community of editors and admins to make such changes to the interface is a *good* thing, but want to be clear exactly what is changeable and by whom, and if off-wiki requests are needed, where to make them, and making this location and the whole roll-out clear to people via on-wiki notices. Presumably, an update will be made to the on-wiki pages about this before it goes live? And there will some site notice giving some warning? having things change mid-edit could be a bit disconcerting! Carcharoth

On Mon, May 3, 2010 at 11:15 AM, Carcharoth <carcharothwp(a)googlemail.com> wrote: These are good points. I'm cc'ing this to foundation-l, as the Foundation is taking the primary role in rolling this out to the English Wikipedia (or at least, has been). Maybe William and Erik can comment on what the expectations are for the extension's documentation, and what sort of delay we can expect between the coming development end point and live access on en.wp? Does the WMF plan to follow the trial roll out described on community pages there, or something else? Nathan

On Mon, May 3, 2010 at 12:48 PM, Carcharoth <carcharothwp(a)googlemail.com> wrote: What was decided on was using the protection control knobs to selectively activate it. Technically this means that nothing changes when this is activated. In practice, however, I expect this means that every semi and full protected page will be fairly quickly converted and in the main namespace actual protection will quickly become a infrequently used drastic measure employed only against high volume trouble making. ... so I would expect it to hit a lot of pages rather quickly as a lot of popular pages are semi-protected. Hopefully the visible difference for most people will be minimal and the primary visible effect will be that popular pages will once again be editable for people without long established accounts. As the software currently stands, however, it generates some rather obnoxious messages advising you that your edits won't be visible until they've been reviewed... but I hope that we get rid of that before launch.

On Mon, May 3, 2010 at 1:11 PM, David Gerard <dgerard(a)gmail.com> wrote: If you haven't caught it— my strongly held and long standing recommendation is that we make the process as invisible as possible: By overloading the cookie that is set when a user (inc. anons) edits we can switch these people over to the draft-by-default view, either in a full-on all-articles sausage making mode like a logged in user, or just for the articles that they've edited. (I know I'm being repetitive on this point, but I'm going to continue making it at least until people start arguing that it shouldn't be done rather than ignoring it or mistakenly believing that it isn't possible) After that is done you could decide to say absolutely nothing at all, you certainly don't have to engage in the whole hopeless task of trying to explain to a newbie that their edit is _still_ available to the whole world but that it isn't yet the default view of the article. I prefer the nothing at all route myself: Let the notice that appears on the top of the draft version tell the story. At least by that point, after seeing the edit live on the site—the risk that the user will believe their edit has been lost forever down a deep dark hole of review hell will be reduced. Of course, there is the risk that the cookie will expire or the user will browse from another before their edit gets reviewed... but thats still less bad than the common case of an over zealous vandal fighter reflexively reverting the user's edit so that its gone for good.

On Mon, May 3, 2010 at 2:29 PM, Anthony <wikiagk(a)googlemail.com> wrote: I'm saying that IF you've edited an article you should continue to see the draft version of that article, even if you're an anon, for at least as long as your session cookie lasts. Our interest in protecting the reader (and the subject of the article via reader exposure) from the vulgarities of the editing process are pretty much non-existence once the reader has stuck his arm in the muck and become an editor. I was also suggesting, as a possible alternative to the above, making anons who have performed an edit get the unvetted version of _all_ articles for the rest of their session. I believe would be somewhat easier to implement and would largely fall under the same rationale as above (e.g. if you've edited ANY article you're already in the top 99-th percentile in terms of understanding the possible fallibility of Wikipedia). If we assume that someone who has edited is far more likely than average to edit again, then sticking them with the drafts is good because they won't have the issue of attempting to make a fix which is already existing in the most current draft version. I guess we'd need to know what percentage of readers have the editing cookie set ... based on text squid object hit rates I know it must be a very low number, but I'm not sure exactly how low. On Mon, May 3, 2010 at 2:37 PM, David Gerard <dgerard(a)gmail.com> wrote: I certainly don't want to lie. But is it any less of a lie to imply that "their work hasn't gone live" when, in fact, it very much has gone live? It's just behind an additional click for the general public, shown by default to tens of thousands of logged in users, and shown to anyone who makes an edit to the article. Is "hasn't gone live" really applicable when, if everything works well, it should be the default for the general public as well typically within minutes— and not the days or weeks that people would seem to assume? If you go look at the early slashdot discussions about flagged revisions there are a multitude of concerns raised about edits being "reviewed" before going live. For example, _many_ people raised the concern that no one would take the effort to merge these changes and that they'd be lost when the next established user makes an edit, others raised the concern that edits would be secretly vanished by the reviewers that flagging would remove the transparency of the current process. When this was in the news I was grilled by some business contacts on this— since they know of me as someone who knows about Wikipedia... When I explained that the unreviewed versions were still available to _everyone_, just not shown as default I got a rather mindblown reaction. It simply doesn't occur to people that you could have an effective review system which isn't predicated on secrecy! So given that a complete explanation will be far too complex and TL;DR to include in a colored notice box— I think saying nothing is significantly less dishonest than any implication that the edit isn't live. Alternatively, simply giving the users a link to a page describing the complete edit life-cycle, "This page is [[protected]].", would be fine as well... those who care could go get a complete understanding, the vast majority who don't care about the minutia of the editing process can comfortably ignore it and not worry that their edit is LESS likely to be used then it use to be.

On 3 May 2010 20:08, David Gerard <dgerard(a)gmail.com> wrote: Once we've had it running for a while, we can give people an idea of how long it will take (hopefully the median will be no more than a few minutes).

On Mon, May 3, 2010 at 3:14 PM, Anthony <wikiagk(a)googlemail.com> wrote: Except it _is_ visible. To the whole blinking world. Do we have a notice on talk pages that your edits will not be visible? Why not? The draft version is more accessible than the talk page (in that the draft version is shown by default to some people). The way the current setup is working— you get a warning "Edits must be reviewed before being published on this page" which is an outright lie unless you admit a domain specific definition of "published" which is mostly inconsistent with common expectations, and if you navigate away from the page and come back it looks like your edits are gone completely. :( On Mon, May 3, 2010 at 3:12 PM, Thomas Dalton <thomas.dalton(a)gmail.com> wrote: That would be pretty darn awesome. We still have the problem of people who don't see / misunderstand the notice and then their edits are missing if they navigate back to the page within the window. It's a pretty sucky experience, even knowing full well how the site works, to navigate back to a page and have all your hard work reverted within a minute. We ought to try as hard as we can to avoid the flagged revision system falsely creating this experience. On Mon, May 3, 2010 at 3:08 PM, David Gerard <dgerard(a)gmail.com> wrote: "Your edit is visible to you and any logged-in users" is still a misleading over-simplification. It's also visible to everyone who clicks two buttons rather than one. What about pretending that it isn't complicated again? ;) "Your edit is in the default view for you and any logged-in users." would be better, hyper-linking default view to a page explaining flagged protection. This would be scrupulously and technically honest, without being excessively long or ringing alarm bells for people who don't really care about the details. Following Thomas Dalton's suggestion, this could be augmented to "Your edit is in the default view for you and any other logged-in users. Most edits are made the default view for all readers within X minutes." once there is data to support the claim. Regardless of the notice part— it sound like you support making anons see the draft version of a page (all pages?) after they've edited? The two issues are somewhat separable— though I think a weakly worded notice requires drafts-after-edit, while a more detailed notice can be used in either case.

On Mon, May 3, 2010 at 3:38 PM, David Gerard <dgerard(a)gmail.com> wrote: So— thoughts on all articles vs the single article you edited?

That would be for *most* IP editors, correct? Because I've run across a few IP editors that seemed to care, even if they don't edit on a consistent basis. Emily On May 3, 2010, at 3:25 PM, David Gerard wrote:

Okay, true. I just wanted those editors acknowledged. That was all. I'm a bit nitpicky, and it appears I've caused a digression. Carry on. Emily On May 3, 2010, at 3:34 PM, Anthony wrote:

At which point, you need to ask "So why not edit it again?" Emily On May 3, 2010, at 3:46 PM, David Gerard wrote:

On Mon, May 3, 2010 at 12:08 PM, David Gerard <dgerard(a)gmail.com> wrote: I think I agree with David. But I think it might be helpful to go through the possible situations. 1. problem: no one understands what's going on. 1a. time should be spent, by someone who understands it -- and then by someone who doesn't to copyedit* -- on writing a very, very, very clear explanation of the flaggedrevs setup that is planned for implementation. Then this page can be linked to in whatever noticebox explanation that may or may not appear on editing. It can also be given to news reporters, because you know the "wikipedia has implemented review" stories are going to start flying once we set this thing up on en: (they already started flying when it was just a proposal). 2. problem: psychology of the anonymous editor: what's the best outcome for a in-good-faith editor? 2a. we already disallow anons from editing semiprotected articles. So in lieu of that, having a box that pops up that says "this article is under protection, and therefore your edit is subject to review" doesn't seem so bad. Note this isn't necessarily a great solution for the entire site, but just for those articles that were formerly uneditable at all by anons. 2b. For editing in general (assuming it ever gets that far), I can think of a few test cases. My sense of the matter is that for experienced editors making a change is not such a big deal; each individual edit neither costs us much or is that important to our experience of the site. But if you are a new editor -- let's say a newly registered account or anon -- each change is worth a lot and is meaningful. I've certainly talked to a lot of folks interested in wikipedia who have told me, proudly, that they have made five edits. For those folks, their five edits are individually each important -- important in their understanding of how wikipedia works and for their sense of being a contributor. That said, I think we need to try and imagine people's behavior around their edits. Would they go and look at the article again later (post session-cookie) to see if their change stuck? I think they probably would. I also think the experience of seeing an edit "go live" is pretty magical. So how we deal with this is dependent on 1, how flaggedrevs works -- but I would think that some sort of clarifying statement -- who the edit is visible to, and where to go to see the version with the edit -- might be nice rather than the impression, on later viewing, that they've been reverted because their edit isn't showing up. 2c. for non-good-faith editors -- vandals -- it might be nice to also have a notice to this affect, to let people know that their edits are being looked at; each vandalism transaction does have a cost and it could be good to let people know that. 2d. Having the data on how long it takes to flag revisions would be nice -- and I suspect that if the trial is started with only protected/semiprotected articles it won't be long at all. If it's only minutes the messages might be slimmed down & Greg's idea of an invisible transaction carries more weight with me. 3. problem: we don't really know how this is going to pan out 3a. I see a lot of conflicting rhetoric about why we want flaggedrevs and what its role is. Indeed, if the goal is to promote wikipedia as more accurate (tm), then I see no special problem about notifying people that their edits are reviewed -- as Anthony says some might welcome it. If we want it to be an invisible process, part of the mysterious inner workings of the site along with template markup and RFCs, then Greg's idea makes more sense. -- Phoebe * for instance, I have no idea what's going on, despite following this thing half-heartedly for years. so I'm pretty sure I could copyedit the documentation from the point of view of a clueless n00b.

On Mon, May 3, 2010 at 4:18 PM, phoebe ayers <phoebe.wiki(a)gmail.com> wrote: [snip] Absolutely. We need to pre-fab the PR on this to the greatest extent possible. Including an announcement of what we're going to do and when, then an announcement of what we've done. But I don't think that any of that interacts much with the fine details of how the implementation works. I think that for usability we need to assume that the user has probably missed all our fantastic announcements and explanations. This is absolutely true. But I think it's completely impossible to explain to a listener who isn't going to invest at least 5 minutes in hearing our explanation. So this is an important point on the PR front, but I don't think we learn anything useful for software usability from it. This is basically the heart of my concern: That one edit is important, that it isn't lost is important. That the site making it disappear will be very upsetting for a least some people some of the time. ... and that no amount of explanatory boiler plate can eliminate that problem because some people will miss it due to banner blindness or simply not understand it. Absolutely. But I would expect and hope that the median review time is minutes, for typical and uncontroversial changes. If it's not— then we need to improve the review process, because we're managing median vandalism revert times less than that. I think what we ought to try to do is to preserve the magic for protected pages as much as we can. Even with a full understanding of the implications— that your change is on the draft page rather than the primary one, it's still pretty magical. This is important for new contributor relations because even if protection is only ever used on the "small number" pages which are currently semi/protected they all tend to be fairly high traffic pages. It's pretty likely that that flagged page behaviour will completely define the new contributors first experience with Wikipedia. As you pointed out— delayed is better than denied, but... Full agreement, I think. I just am concerned that we retain the kind of positive outlook that you've presented here rather than the negative outlook which focuses on how the contributor is being restricted. [snip] I don't think the goal of "More accurate™" is in conflict with "Maximally inclusive in whom is allowed to edit". Through the power of the default-view and the power of transparency we can have _both_, and I think the community has demanded a system which provides as much. The challenge here is that the initial impression for "review" and "More accurate" is a secretive, restrictive, controlled, and slowly moving system... largely because this what traditional mediums provide. While "inclusive" is viewed as a crazy anything-goes anarchy (which was never really applicable to Wikipedia, even before protection). I think the message we need to express is that we're trying to combine the qualities of both extremes into a moderate composite which is even closer to the radical openness of the early Wikipedia while simultaneously being more accurate than the current system. I don't know how to craft a PR message around this because to an outsider it sounds impossible for exactly the same reason that the whole idea of Wikipedia sounds impossible. People are very quick to jump to the 'restrictive' understanding because it makes Wikipedia finally make sense: "See! radical openness really doesn't work!"

On 05/03/2010 06:13 PM, phoebe ayers wrote: Great question! There are two broad sorts of testing typically done for things like this: qualitative and quantitative. For the qualitative approach, you'll sit some representative user down and have them perform some tasks. You sit quietly and watch them carefully, often recording both the screen and their faces. Some people like to have them narrate their actions and thoughts, so you can get a better idea of how they're taking things. Advance versions even include eye-tracking, so you can get better insight into what they're looking at and reacting to. This can be done in person or remotely, and there are sites like usertesting.com to automate the process. There are a number of quantitative approaches, but the most popular is A/B testing, where we'd deploy two different versions of the feature and assign a bunch of users to each group. Then we'd look at their behavior over time with respect to certain metrics, like returning to look at their articles, talk page participation, and long-term editing behavior. Right now the WMF is doing a little of the first, and -- AFAIK -- none of the second. The qualitative testing is relatively expensive and time consuming to do. For a minor variation like this, qualitative testing is probably better done as part of a broader test of novice editing experiences. I'd love to do more quantitative testing, as that would settle a lot of questions like this, but being set up for that would require non-trivial infrastructure changes, plus a serious discussion about how much cookie-based user tracking we want to do. Right now there's other infrastructure work that is definitely higher priority. E.g., making sure that a well-placed hurricane won't cause major site issues. So for now, although I wish it were otherwise, we may have to settle for trying things out live and trying to calibrate our gut feelings based on what user reactions we can glean. William

On 3 May 2010 17:59, Gregory Maxwell <gmaxwell(a)gmail.com> wrote: On 05/03/2010 11:25 AM, Gregory Maxwell wrote: We discussed this at some length today, and I wanted to update everybody. From our perspective, there are two parts to this. One is using cookies to make it so that anons see their edits when looking at pages where they have pending edits. The other is de-emphasizing (or at the extreme, concealing) the fact that some edits will require approval. We think the first part is a great idea, but not crucial for launch. So we've added to the backlog, and barring some sort of excitement that demands more immediate attention, we'll get to it soon after launch. For the second part, we're concerned. There are reasonable arguments on both sides. It's a solution to a problem that's currently only hypothetical, that anons will be put off if we're clear about what's actually going on. Even if there is some effect like that, we'd have to weigh that against our project's strong bias toward transparency. And if despite that we decided it was worth doing, we still think it's not crucial to do before launch. So given all that, we think it's better and wait to see if hypothetical problem becomes an actual problem; at that time we'll have better data for deciding how to handle the problem. As mentioned earlier, the Foundation is committed to supporting the experiment with further development as needed, so resources are already allocated if this becomes an issue. We hope that everybody is at least provisionally satisfied with that, and I'd ask those most worried about this to keep an eye on things after launch and if necessary raise the issue again, optionally with a well-earned I-told-you-so. William

On Thu, May 6, 2010 at 5:22 PM, William Pietri <william(a)scissor.com> wrote: Who is the we in your message? (I'm just asking because its entirely ambiguous, since you were quoting me it looks like I was involved, though I obviously wasn't :) ) [snip] When is this launch planned to occur? I'm really saddened by the continued characterization of de-emphasis as something which reduces transparency. It has continually been my position that the current behaviour actually reduces transparency by being effectively dishonest on account of being an excessive over-simplification. The message currently delivered by the software is: "Edits must be reviewed before being published on this page. " And yet the edit will be instantly available to every person on earth (well, at least all of the people who can currently access Wikipedia) the moment it is saved. The interface text is misleading. It is not a good example of transparency at all. If I knew how to fix it, I'd suggest alternative text. Sadly, I don't— I think that any message which conveys all of the most important information will be too long and complicated for most people to read. What I was attempting to propose was being selective in which of many possible misunderstandings the contributor was most likely to walk away with, this isn't the same as proposing a reduction in transparency. What is the plan for collecting this data? What metrics will be collected? What criteria will be considered a success / failure / Problematic / non-problematic? I have seen many tests invalidated the bias of having seen the outcome: "Fantastic success! We have a customer retention rate of 60%!" "What would have been failure?" "um..." "Didn't you say last week that you were expecting that we retained almost everyone?" "but! 60% is a _good_ number!" I'm unsure how we could measure the effect of the interface here outside of a controlled usability study. I suppose we could take the set of new contributors group them by their first edit being on a flagged-protected page or not, and then for each group count how many of the users contributed a second time. Then we would decide that the interface is not problematic if the two groups are statistically indistinguishable, and that it may be problematic if the first-flagged-protected users were less likely to contribute a second time by at least some confidence level. But this I think this would require non-trivial software infrastructure to track client identities for anons which we currently do not have. There are also many other methodological problems with this approach, — e.g. Making the reasonable assumption that flag-protection will largely replace semi-protection articles of certain types (e.g. Bios) will be far more likely to be flag-protected, we could reasonably expect that the characteristics of these contributors to differ, that they'd be less or more likely to contribute again, regardless of the flag-protection. Or, in other words, making a reliable measurement of this may be a project of compatible complexity to this whole deployment. ... which is why my initial suggestion was to take what I believed to be the most conservative approach in terms of editor impact— simply because the cost of actually measuring the result is likely too great in comparison to the benefit of being less conservative. I apologise for harping over these details, but for years the people promoting a little review have dealt with opposition on the basis that adding any friction at all may significantly discourage contribution, and none of us want to slay our golden goose— the constant stream of new contributors. This is a concern which I've heard clearly expressed by Sue and Erik, to name some of the most notable examples— it's not merely an example of David's "unstoppable petty complaint". While I've personally dismissed this concern I've done so because I consider getting a little review into the process to be _essential_ for the project's future and thus worth even considerable risk, not because I believe it to be unfounded. By my reasoning, we've only got one shot at this. If the process significantly dampens contributions I don't believe it will be possible to convince the English Wikipedia community to tolerate another attempt in the next several years, even if we think we know the cause of the failure and think we can resolve it... since it will be impossible to prove these things, and nothing short of proof or public shame can reliably swing the community inertia. Accordingly I want to make sure that we're making every reasonable concession to ensure that the flagging does not discourage contribution and that if it fails that it fails only because the process of review is essentially incompatible with our development model and not simply because some poor interface decisions doomed it to failure. Cheers,

On 05/06/2010 04:38 PM, Gregory Maxwell wrote: Sorry, I mean we the project team. Howie Fung, Aaron Schulz, Rob Lanphier, me. We do a Skype call every Thursday. The most accurate answer: when it's ready. Qualitatively, soon. Those who want a more precise answer are welcome to extrapolate from the data here: http://www.pivotaltracker.com/projects/46157 We are entirely open to improvements to that text, or any other text. It's the best we've come up with so far, but we're under no illusions as to its perfection. We do think it's important to indicate to people that if they browse back later, they will not see their change. It will also not be visible to the general public, including if they tell friends to take a look, and we want to be clear about that. Sorry if I misunderstood! I thought somebody was arguing for removing the message entirely and just showing them the latest draft as if it's the version that everybody sees, but I must have gotten that wrong. As I mentioned elsewhere in the thread, I would be tickled pink if we could collect the kind of serious data I'm used to using in judging features like this. Our software and production environment do allow for that currently, and there are higher-priority infrastructure needs that must come first. I would love for the usability team to user-test this, but they'll have to do it as part of a series testing the anonymous editing experience more broadly. I'm not sure what their schedule is, and for the moment it doesn't matter; if I were to delay launch for that people would rightly have my head on a pike. So until then, we'll have to use the low-grade plural-of-anecdotes sort of data that drive a lot of Wikipedia decisions. Looking beyond the Foundation's resources, there's nothing stopping volunteers from user-testing this, and I would be over the moon if volunteer developers took up the cause of A/B testing. I expect it will take a while to get that in to a long-lived codebase like Mediawiki I'm sure there are plenty of smaller Mediawiki installs that could benefit from A/B testing even before we're ready to deploy something like that on the Wikipedia production cluster. I hasten to add that we will be collecting other sorts of data. You can see that on labs here: http://flaggedrevs.labs.wikimedia.org/wiki/Special:ValidationStatistics We are also fortunate to have some time from the mighty Erik Zachte. Right now he's looking at dumps from the German use of FlaggedRevs to come up with useful stats so that he'll be ready to look at the English Wikipedia usage during the experiment. We also expect that we'll be improving the validation statistics page based on feedback both from him and involved users. William

On Thu, May 6, 2010 at 7:38 PM, Gregory Maxwell <gmaxwell(a)gmail.com> wrote: How about this. No message on the edit page itself. When they save the edit, they're redirected to the draft page of that article, with a message at the top saying something like "This is a publicly-viewable draft, and will be shown to all viewers by default after review." There's no need to mention it *before* the edit, is there? Mentioning it after the edit shouldn't discourage contributions too much. To the contrary, if it shows up on the default page, they'll probably be happier than now, knowing that someone took the time to explicitly declare their edit worthy. If it doesn't, no different to them than if it was reverted under the current system. I'm not sure that making the edit experience exactly the same as now is best. It would confuse people who view the page from another computer shortly after editing, before the edit is approved, and assume that it was rejected.

agreed on that. I'm very apprehensive about the possible negative effect on new contributors, but this seems a good solution David Goodman, Ph.D, M.L.S. http://en.wikipedia.org/wiki/User_talk:DGG On Fri, May 7, 2010 at 8:36 PM, Gregory Maxwell <gmaxwell(a)gmail.com> wrote:

These are great questions, and we're actually having a big meeting about the project this afternoon, so I'll be sure to raise them to make sure we all have the same notion. That said, a few of quick responses from my perspective: On 05/03/2010 08:15 AM, Carcharoth wrote: There's the technical part of this and the community part of this. My understanding is that the technical side of the rollout is well understood, and that our substantial time in the labs environment means we are not expecting major problems. I also am given to believe that if there are major problems, rolling back will not be a big deal. That will get us to having the feature enabled, but not in use. That next leg is mainly up to the community. Once the software is enabled, any admin will be able to turn on flagged protection for any page, just as they are now able to turn on full protection. I expect there will be a period of experimentation and vigorous discussion to discover exactly when that is a good idea. Once it's in use on particular pages, there's the question of who does the reviewing, how much is needed, and how we make sure it gets done in a timely fashion. Most of that is up to the community as well, and part of the purpose of this experiment is to figure that out as well. From a technical perspective, there are a couple different approaches to deciding who has reviewer powers; in the next week or so I want to start a community discussion on the right model, but we need a little more internal discussion to be able to clearly present those options. As far as the "making adjustments as needed", the plan is that we will absolutely learn things after release, and some of those things will probably require code changes. There is also a list of nice-to-haves that we can do if nothing else more pressing comes up. So work will continue as before, with frequent releases either to production or to the labs environment as appropriate. Once that work tapers off, I'm sure there will be a discussion of where best to allocate resources, but that hasn't even been mentioned yet; the Foundation is definitely committed to supporting this experiment. This is an area where I'm personally a bit ignorant, so I'll be sure to ask. I know that some parts of the interface definitely require a developer to change code and release it. I know that some, possibly all purely textual changes can in theory be done hot, but I don't know who has the mojo to do that on the English Wikipedia. If somebody here knows that, please speak up. My belief, which I will double-check, is that releasing the software will have little or no impact on the editing experience; it's only when an admin activates it on a particular page via the protection interface that the editing experience will change. William