[Wikitech-l] 2017-12-06 Scrum of Scrums meeting notes

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-12-06?veaction=edit&… = *2017-12-06* = == Callouts == * Operations blocked on https://phabricator.wikimedia.org/ <https://phabricator.wikimedia.org/T172025>T172025 <https://phabricator.wikimedia.org/T172025> (Flow) * Operations DBAs s8 master switchover programmed for 9th January * Reminder! This is your last few weeks of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th. ** MediaWiki 1.30 is about to be released in the next few days, so last call for any blockers, phabricator tag is *#mw-1.30-release * Wikidata welcomes secondary review from somebody with knowledge about the recent "section editing" change: https://phabricator.wikimedia.org/T181807 * Have a look at newly initiated https://www.mediawiki.org/wiki/Manual:Coding_conventions/SVG. Feedback welcome! == Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: ** Releasing 5.7.2 (high priority fix for https://phabricator.wikimedia.org/T69015 ) ( https://phabricator.wikimedia.org/tag/ios-app-v5.7.2/ ) ** Continuing work on 5.7.3 (Faster article loading, other minor enhancements) for release before the end of the year ( https://phabricator.wikimedia.org/project/view/2913/ ) ** Continuing work on 5.8 (Reading Lists) for release next year ( https://phabricator.wikimedia.org/project/view/3131/ ) ==== Android native app ==== * Blocked by: * Blocking: * Updates: Releasing beta 2.6.207 ==== Reading Web ==== * Blocked by: * Blocking: * Updates: ==== Reading Infrastructure ==== * Blocked by: * Blocking: * Updates: ** ReadingLists performance improvements ** Wrapping up media + summary endpoints ** References needs a bit more design work ==== Multimedia ==== * Blocked by: N/A * Blocking: N/A * Updates: 3D nearly ready to go, probably before holidays, else shortly thereafter. Also working on prototyping for first feature of Structured Data on Commons. ==== Discovery ==== * Blocked by: * Blocking: * Updates: ===== Maps ===== * Blocked by: N/A * Blocking: N/A * Updates: ** Maps mediawiki integration (Kartographer) improvements #1 on community wishlist: https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey/Tracking ** Redeploying Kartotherian ** Tracking down conflicting versions + NPM === Community Tech === * Blocked by: * Blocking: * Updates: === Contributors === ==== Editing ==== * Blocked by: * Blocking: * Updates: ==== Parsing ==== * Blocked by: Services (because of Cassandra3 migration) on deployment of HTML version bump (for section, figure-inline, html5-id, interwiki-link changes). This is just an FYI since Services & us have already synced up about this. * Blocking: Reading Infra on section tag markup * Updates: ** itwiki, dewiki, and 170 small wikis got switched from Tidy to RemexHtml yesterday - itwiki have flagged a bunch of new issues that hadn't been caught by linting so far. Investigation ongoing and might introduce one or two newer categories to aid editors -- we expect these categories to be sparsely populated ** Templatedata related fixes being deployed today -- For pre-existing transclusions, this will prevent Parsoid from normalizing parameter order to templatedata format ==== Global Collaboration ==== * Blocked by: MW core (or Parsing?) for code review on https://gerrit.wikimedia.org/r/#/c/392990/ and surrounding stack of commits * Blocking: ops for Flow dumps * Updates: ** New tags for https://phabricator.wikimedia.org/ <https://phabricator.wikimedia.org/T167656>T167656 <https://phabricator.wikimedia.org/T167656> are on this week's train ==== UI Standardization ==== ** No OOUI release this and upcoming weeks * Ongoing: ** OOUI & based products: *** icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide https://phabricator.wikimedia.org/T177432 finishing up ** Unify SVG markup across Foundation products https://phabricator.wikimedia.org/T178867 ** Continuous work and per-project SVGO based optimizations, also initiated https://www.mediawiki.org/wiki/Manual:Coding_conventions/SVG == Technology == === Analytics === * Blocked by: * Blocking: * Updates: ** The prometheus druid exporter (to report druid metrics to graphana via prometheus) got some attention and others (outside WMF) will be using it ** First productionized version of EventLogging backend on hadoop launched this week. This means SQL-friendly tables on hadoop with eventlogging data, and for the first time, ability to join to all mediawiki databases in one query ** Wikistats 2 APIs (like Pageview API, but for edit data), alpha launch: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Wikistats ** Wikistats 2 UI, alpha launch: https://stats.wikimedia.org/v2/ ** Decommissioning old DB hosts for EventLogging done, remember that eventlogging DB is now only on analytics-slave, not on analytics-store anymore ** Work on kafka jumbo cluster ** Rebooting one druid host for maintenance brought up unexpected issues with druid and zookeeper, working on that === Cloud Services === * Blocked by: * Blocking: * Updates: ** Annual Toolforge survey closed. 141 responses received (11% response rate). ** Team in Austin this week doing an offsite + attending KubeCon conference ** Will upgrade labpuppetmaster* to v4 puppet packages after the offsite *** Presuming everything in eqiad goes well next week === Fundraising Tech === * Blocked by: None * Blocking: None * Updates: ** New CiviCRM Import tool for Major Gifts ** Paying down some tech debt in Central Notice ** Fixes for fundraising vagrant role ** Continued improvements to Fundraising Dashboard ** Continuing to support FR-non-tech during the fundraiser === MediaWiki Platform === * Blocked by: N/A * Blocking: Global Collaboration for code review on https://gerrit.wikimedia.org/r/#/c/392990/ and surrounding stack of commits * Updates: ** MediaWiki 1.30 Release Candidate 0 released ** MCR and Actor table reviews ongoing ** Comment table: Tried to enable WRITE_BOTH mode on testwikis, but had to revert because it broke CentralAuth and ForeignDBViaLBRepo. *** Will likely wait until the schema change is done everywhere before we move anything to WRITE_BOTH. ** TemplateStyles: ParserOutput stateless transforms got merged. Some Wikibase issues that are being dealt with. ** Firejail: will be deployed for Score ** New PSR-4 autoloader patchset https://gerrit.wikimedia.org/r/373626 ready for review ** PoolCounter for ORES client https://gerrit.wikimedia.org/r/ <https://gerrit.wikimedia.org/r/394407>394407 <https://gerrit.wikimedia.org/r/394407> ready for testing ** MP3 transcoding deployment ** Discussions with third party MediaWiki developers regarding extension deployment/dependency management (composer) === Performance === * Blocked by: N/A * Blocking: N/A * Updates: ** Significant new performance testing documentation in progress (aimed at developers), expected to be delivered before end of 2017 ** New varnish slow log configuration in final testing ** perf testing of cross-data center database writes ** Q3 Goals planning: if you think you might need anything from the perf team, hit me up. ** docker-based MW dev environment (in collab with WMDE) === Release Engineering === * Blocking: ** Scoring platform, https://phabricator.wikimedia.org/ <https://phabricator.wikimedia.org/T181661>T181661 <https://phabricator.wikimedia.org/T181661> *** New scap release planned for this week that will hopefully address this issue. * Blocked by: ** none * Updates: ** [MW Train] Reminder! This is your last few weeks of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th. *** *#mw-1.30-release *MediaWiki 1.30 is about to be released in the next few days, so last call for any blockers people might have...you know...forgotten * === Research === * Blocked by: * Blocking: * Updates: === Scoring Platform === * Blocked by: ** Parallel scap SSH issue https://phabricator.wikimedia.org/T181661 * Blocking: * Updates: ** Outage on Nov 28, incident report is going out today. ** SimpleWiki deployed to ORES *** FYI: Global Collab, please announce and update RC Filters. The extension is only configured on the beta cluster. ** testwiki (RevIdScorer) includes thresholds now, which makes extension development easier, against the real server or in either of our vagrant roles. ** ORES Ext. Refactoring ** Incoming basic support of Icelandic (No RC Filters yet) === Search Platform === * Blocked by: * Blocking: * Updates: * Working on improvements to LTR training * Reindexed wikis to enable improved katakana/hiragana mapping https://phabricator.wikimedia.org/T179945 * Improving completion suggester interaction with namespaced prefix search https://phabricator.wikimedia.org/T178474 * Wikidata descriptions indexed, working on fulltext search * Working on porting Selenium tests from Ruby to JS * Logstash upgrade to ElasticSearch 5.5 on Thu, completing the 5.5 upgrade === Security === * Blocked by: * Blocking: * Updates: ** Lots of development on Phan security plugin by Bawolff ** Maintanenace work on security alert configuration (Github, NSP) ** Reviews: *** Ex:WikipediaExtracts *** Next set of security reviews will be scheduled this week === Services === * Blocked by: none * Blocking: Parsing till early next week * Updates: ** Round of Cassandra 3 bootstraps is complete ** htmlCacheUpdate for wiktionaries are on kafka queue === Technical Operations === * Blocked by: ** Global Collaboration on Flow dumps https://phabricator.wikimedia.org/ <https://phabricator.wikimedia.org/T172025>T172025 <https://phabricator.wikimedia.org/T172025> * Blocking: ** None * Updates: ** Q3 goal planning started, if teams have dependencies on Ops for next quarter, reach out ** Part of DC ops in Singapore for eqsin (new caching DC's name) rollout ** s8 (wikidata) work ongoing, work ontrack. ** Operations DBAs s8 master switchover programmed for 9th January == Wikidata == * Finally got rid of the manual "Wikidata" build process, Wikidata is part of the weekly deployment now: https://phabricator.wikimedia.org/T173818 * Introduced a custom, "compact" entity diff serialization in the wb_changes table. Please look out for side-effects: https://phabricator.wikimedia.org/T113468 * First patches demo the possibility to persist statements on sub-entities utilizing Wikibase's standard wbsetclaim API (namely Forms on Lexemes): https://phabricator.wikimedia.org/T163724 * Core changed the way it handles section editing. Wikibase hooks into this feature, and now behaves unexpected. Review welcome: https://phabricator.wikimedia.org/T181807 == German Technical Wishlist == * Blocked by: * Blocking: * Updates: