[Wikitech-l] Dev Summit ideas from 2013

Hi everyone, In 2013, WMF was still holding it's traditional All-Hands meetings in September. In the days before the All-Hands, we had "Tech Days", which was two days exclusive to WMF Engineering. ArchCom was new back then. That was where we were starting to plan the first Architecture Summit in January 2014. I can't remember if I sent the email below to this mailing list or if I merely squirreled it away on mediawiki.org in February. Nonetheless, it's worth rereading now, since we're just about at the same point (chronologically) in the planning process for WikiDev17 as we were in planning for Architecture Summit 2014. I've posted links to everything I'm talking about above here: <https://mediawiki.org/wiki/Talk:WikiDev17> ...and that place may be a better place to reply than here on list. Rob (p.s. thank you whoever fixed the Talk:WikiDev17 link) ---------- Forwarded message ---------- From: Rob Lanphier &lt;robla(a)wikimedia.org&gt; Date: Fri, Sep 6, 2013 at 5:01 PM Subject: Architecture discussion next week To: Development and Operations Engineers &lt;engineering(a)lists.wikimedia.org&gt; Hi everyone, I’d like to bring you up to speed on what we’re thinking about for the Architecture discussion at the tech days next week. I have a long background written up below, but I’d like to lead with the agenda. The overall session is 90 minutes, and a lot of ground to cover: * 15 minutes - Background: Basic explanation and clarifying discussion of the process * 15 minutes - Role of our architects, senior developers, WMF in the process * 15 minutes - Running through a couple example RFCs (no decisions, but highlighting the discussion; see below if you want to volunteer) * 15 minutes - Brainstorming on RFCs that don’t exist yet, but should * 10 minutes - How do we move things faster (e.g. open invite weekly meetings?) * 10 minutes - Architecture summit in January * 10 minutes - Summarize next steps; assign action items Below is the way-too-long explanation about all of this. I’ve broken this up into sections which mostly mirror the agenda above. ==================== Background --------- The way big architecture decisions get discussed and eventually implemented for MediaWiki is highly dependent on context. Much of the work is very driven by big features; for example, we made changes to the way the job queue works in service of Echo (with many benefits to other aspects of the system, such as how we handle video transcoding). Other work is driven by community development; for example, IAlex’s work on the context object[1]. There’s still other work that people would like to embark on, but may not see a clear path forward, so they work around problems rather than address them. In all of these cases, developers are often unclear about what they “should” do, so they muddle through going with what worked for them in the past. At the Amsterdam Hackathon, we started the process documenting our architectural guidelines[2][3]. A goal of this documentation is to have some written guidelines to compare design proposals against, rather than basing our decisions on the design aesthetics of whichever reviewer happens to be looking at a big change in Gerrit. It has served as a means of having discussions some seemingly intractable disagreements and has defined a way forward for experiments in more controversial areas. For example, the idea that we should break up objects into value objects and logic objects is something we’re still divided on, if our discussion in Hong Kong is any guide[5]. However, rather than continuing this argument in the abstract, we agreed that a specific example will be helpful, so Daniel Kinzler is working on an example based on the Title object. Our defined process for making big changes to MediaWiki is the RFC[4]. We’ve had this process for a number of years, but until recently, we didn’t have a concrete commitment to move anything forward. Even now, the process is still a bit fuzzy, but things are moving forward. Tim Starling did a very preliminary assessment of all of the RFCs in the queue, and some have moved forward. We’re now at the point where we need to have a conversation about how to make this really work for us, both as WMF Engineering and as a larger development community. We seem to have general agreement that MediaWiki could use more coherence in its design, but we’re still quite far from agreement on exactly what we should aspire to. ==================== Role of our architects, senior developers, WMF in the process --------- We’ve gotten big enough as an organization that having just one or two people (e.g. Tim and Brion) make all of the major decisions about MediaWiki architecture just doesn’t scale, at least not the way we’re doing it today. But, as noted above, it’s also important that we strive for some level of coherence in how we do things, and a very common strategy is to centralize the decision-making in a single BDFL (e.g. Linux or Python). We need to have a discussion about what makes sense for the way we do development. ==================== Example RFCs --------- I haven’t yet identified which RFCs would be best to talk about. The Front End folks are planning to talk about the LESS RFC in an earlier session, which might be useful for them to report on rather than talk about in detail. I’m open to suggestions, especially if you are planning to be here and have one that you feel comfortable quickly presenting to the group (lightning talk style). ==================== Brainstorming on RFCs that don’t exist yet, but should --------- I think there’s a lot of things we know need to change about MediaWiki, but isn’t actually documented in RFC form. Additionally, there’s probably work that’s already underway that could stand a design review. Let’s just have an etherpad open and hammer out a list together of things that need RFCs. We can maybe put the result of that on a page titled “RFC/Incubator” or something like that. ==================== Moving things faster --------- The architects generally have wanted to make the time to get together, but circumstances have intervened keeping that from happening. It may be helpful to have a more open process to make review happen faster (e.g. a weekly open-invite meeting). Please come prepared with your thoughts on what might work, and more importantly, what you personally are prepared to volunteer for to make things move more quickly. ==================== Architecture summit in January --------- We’ve spent some time this summer in hazy planning mode for an architecture summit in January. The basic structure of the architecture summit would be: * Put out a call for RFCs for badly needed architectural changes, with a deadline some small but reasonable number of weeks beforehand * Cherry pick the most important RFCs for discussion resolution * Spend a reasonable amount of time discussing each RFC, attempting to come to consensus about acceptance of the work, or at least about next steps. The goal of all of the above is to have the architectural conversations that are just too hard to have over email and IRC, which are manifold. This will be the place where we can chart a multi-year course for MediaWiki development. This is somewhat similar to what (I think) the Linux kernel developers do at the Linux Kernel Developers Summit[6] (I haven’t attended, so I’m mainly relying on third-party accounts). One thing that’s been difficult for us to figure out is what the right way of picking the attendees for this. This is far from a settled question, and I think it’d be healthy for us to discuss how this should work next week. ==================== Conclusion ---------- Not much more to say here, other than linking to the agenda, still subject to change: https://www.mediawiki.org/wiki/Architecture_guidelines/Meetings/WMF_Enginee… I’m looking forward to talking more about this next week! Rob [1] https://www.mediawiki.org/wiki/Requests_for_comment/Context_object [2] https://www.mediawiki.org/wiki/Architecture_guidelines [3] https://www.mediawiki.org/wiki/Architecture_guidelines/Meetings [4] https://www.mediawiki.org/wiki/RFC [5] https://www.mediawiki.org/wiki/Architecture_guidelines/Meetings/Wikimania_2… [6] https://en.wikipedia.org/wiki/Linux_Kernel_Developers_Summit