[Wikitech-l] 2017-12-13 Scrum of Scrums meeting notes

*https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-12-13 <https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-12-13>* *= **2017-12-1**3** =* == Callouts == * s8 database master switchover scheduled for 9th January https://phabricator.wikimedia.org/T181645 [please keep in callouts until Jan 9th] * LDAP update for user thiemowmde needed: https://phabricator.wikimedia.org/T181130 * Kafka jumbo cluster (eqiad) ready for testing connections via TLS. This new version of kafka enables consumption via given timestamp - Discovery will use it for wdqs. Aiming to switch some of our clients this week. == Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: ** Released 5.7.2 (high priority fix for https://phabricator.wikimedia.org/T69015 ) ( https://phabricator.wikimedia.org/tag/ios-app-v5.7.2/ ) ** Continuing work on 5.7.3 (Faster article loading, other minor enhancements) for release before the end of the year ( https://phabricator.wikimedia.org/project/view/2913/ ) ** Continuing work on 5.8 (Reading Lists) for release next year ( https://phabricator.wikimedia.org/project/view/3131/ ) ==== Android native app ==== * Blocked by: * Blocking: * Updates: * Released 2.7.221 to production ** fully configurable feed, On This Day cards in feed, new Randomizer, new Black theme. * Continuing to work on performance improvements with large numbers of reading list pages. ==== Reading Web ==== * Blocked by: * Blocking: * Updates: * Still working on chromium service. Almost finished performance testing. * Re-enabling page previews A/B test to collect data to influence a decision on performance. ==== Reading Infrastructure ==== * Blocked by: * Blocking: * Updates: ** Reading lists is in production ** will look at patrolling support for WP0 piracy next ** Media endpoint pretty getting ready to be exposed ** Finalizing touches for summary endpoint (in Beta Cluster) but may need more adjustments for when to return 204 (No Content). ** Getting close for references endpoint ==== Multimedia ==== * Blocked by: N/A * Blocking: N/A * Updates ** 3D: A few changes to be made, on track to deploy sometime soon after the holidays. ** MediaInfo: Prototyping and wireframing currently underway for first features. ==== Discovery ==== * Blocked by: * Blocking: * Updates: ** lots of new portal automation documentation https://gerrit.wikimedia.org/r/#/c/396407/ ===== Maps ===== * Blocked by: N/A * Blocking: N/A * Updates: ** Frontend Mediawiki maps integration came in #1 in Community Wishlist: https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey/Tracking ** Other: Paul Norman re-elected to OpenStreetMap Foundation board === Contributors === ==== Editing ==== * Blocked by: * Blocking: * Updates: ==== Parsing ==== * Blocked by: * Blocking: * Updates: ==== Global Collaboration ==== * Blocked by: Security on re-review of https://phabricator.wikimedia.org/T144467 * Blocking: Ops on Flow dumps still; Matt comes back on Monday and he's the expert * Updates: ** Nothing of note, mostly been fixing regressions recently ** Matt comes back on Monday (Dec 18) ==== Contributors Design ==== * Blocked by: * Blocking: * Updates: ==== UI Standardization ==== ** No OOUI release this and upcoming weeks till January * Ongoing: ** OOUI & based products: *** 'constructive' flag in has been dropped entirely, please replace code remainders with 'progressive'. Will go in effect in 1st release 2018. *** icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide https://phabricator.wikimedia.org/T177432 finishing up *** Consider changing :hover tools and menu background to use Accent90. https://phabricator.wikimedia.org/T166560 ** Unify SVG markup across Foundation products https://phabricator.wikimedia.org/T178867 ** Continuous work and per-project SVGO based optimizations === Community Tech === * Wrapping up our survey, tightening loose ends from this year. == Technology == === Analytics === * Blocked by: * Blocking: * Updates: ** Load testing druid prior to launch of wikistats 2.0 - It provides good enough concurrentcy and response times. Last snapshot of data for wikistats2 is of the quality we like. Aiming to announce launch this Thursday: https://stats.wikimedia.org/v2/#/bm.wikipedia.org ** Working on new set of APIs to power map visualizations, pageviews per country monthly ** Kafka jumbo cluster ready for testing connections via TLS, this new version of kafka enables consumption via given timestamp. Discovery will use it for wdqs. Aiming to switch some of our clients this week ** Notebook1002 got recommissined as a kafka host - Almost there (hard drives issues) ** Superset is productionized - We are testing it internally in the teasm and will anounce soon === Cloud Services === * Blocked by: * Blocking: * Updates: === Fundraising Tech === * Blocked by: nothing * Blocking: nothing * Updates: ** Fundraiser going pretty smoothly (knock wood) ** Trawling logs for errors to fix ** Investigating potential banner impression shortfall for WMDE ** Getting some civi stuff merged upstream ** fixes & enhancements to internal dashboard === MediaWiki Platform === * Blocked by: * Blocking: * Updates: ** MediaWiki 1.30 Released ** code review: *** https://gerrit.wikimedia.org/r/#/c/392990/ and https://gerrit.wikimedia.org/r/#/c/392991/ for Global Collaboration *** MCR patches ** Actor table patches are almost ready to merge ** script to clean up imported usernames ran on a limited set of wikis with no errors so will be run on all wikis ** PSR-4 autoloading support merged ** namespacization and firejail work continues ** went through some bug backlogs including incident response, submitted some small changes ** two meetings with representatives of Gamepedia (owned by AWS/ twitch.tv/curse) ** Tim: will go to TechCom meetings on Wednesday afternoon, then on vacation after that until January 1 === Performance === * Blocked by: n/a * Blocking: n/a * Updates: ** NavTiming extension updates should be going out quite soon. Don't expect any impact on other teams. ** ChronoogyProtector caused problems last week, esp. for Wikidata. Fix is in place, many thanks to the folks who jumped in to help out with that. ** mcrouter puppet config waiting on additional code review. May need some additional review cycles from TechOps. ** ProxySQL SSL termination - may need more time from Jaime, not yet sure. ** Series of blog posts on Thumbor migration published to the main WMF blog ** WebPageReplay now working with Firefox. === Release Engineering === * Blocked by: n/a ** MW Platform team: https://phabricator.wikimedia.org/T182344 - "Blocker must be a local user or a name that cannot be a local user" * Blocking: n/a ** Blocking ORES with some scap issues. * Updates: ** [MW Train] Reminder! This is your last week of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th. ** [Meta] Q3 goal planning ** [Security/Ruby] T180878 Upgrade RuboCop and Rubyzip (Ruby) *** All done except for Minerva (needs a +2) and mediawiki/debian (unsure if needed). ** [Security/Jenkins] Upgraded a bunch of Jenkins plugins Monday morning EU time (after a bunch of security releases). ** [Phabricator] Exploring the use of Selenium tests for search quality regressions ** [TechDebt] Getting further on the definition of “steward”. We will be talking with Victoria and Toby “soon”. ** [SSD] One more merge (*https://gerrit.wikimedia.org/r/#/c/395570/* <https://gerrit.wikimedia.org/r/#/c/395570/)>) <https://gerrit.wikimedia.org/r/#/c/395570/)> and our quarterly goals are complete. ** [PostMortem] ORES post-mortem completed last week: *https://etherpad.wikimedia.org/p/Post-Mortem-T181006* <https://etherpad.wikimedia.org/p/Post-Mortem-T181006> === Research === * Blocked by: * Blocking: * Updates: === Scoring Platform === * Blocked by: ** Blocked on releng, by scap issues. We're getting lots of help. Unfortunately, this is affecting Q2 goal of ours (deploying to new ORES cluster). * Blocking: * Updates: ** Reverted models about to go out for iswiki and eswikiquote *** Need an ops merge for https://gerrit.wikimedia.org/r/#/c/398078/ *** git-lfs -- super excited. Seems like there's not much left before we can use it. === Search Platform === * Blocked by: none * Blocking: none * Updates: * New blog post about failed queries: https://blog.wikimedia.org/2017/12/12/failed-queries-fear-of-missing-out/ * Fixes for completion suggester merged, to be deployed soon https://phabricator.wikimedia.org/T178474 * Added tuning for wikidata prefix search https://phabricator.wikimedia.org/T182136 * Porting to browser tests to Selenium mostly done, setting up CI runs * Categories in WDQS are now auto-updated weekly from dumps (see https://www.mediawiki.org/wiki/Wikidata_query_service/Categories ) * Working on improvements to LTR training * Working on tuning fulltext search for Wikidata === Security === * Blocked by: * Blocking: * Updates: * Security plugin for phan has been announced and published (yay!) * ** Reviews: *** Google MT *** mediawiki-services-chromium-render *** stacktraces on wikis *** git mirroring to diffusion === Services === * Blocked by: none * Blocking: none * Updates: ** Parsoid storage for wikipedias moved to Cassandra 3. Please report anything weird you notice with VE ** Parsoid and MCS content-type bumped ** HTMLCacheUpdate for all non-wikipedia projects processed on Kafka === Technical Operations === * Blocked by: * Blocking: * Updates: ** Asia DC (eqsin) on site installation complete ** Monitoring of production k8s cluster setup ** More work on porting system metrics to Prometheus ** HHVM on API appservers occasionally gets stuck - investigation in https://phabricator.wikimedia.org/T182568 == Wikidata == * Good progress on opening the default Wikibase API for custom entity types (namely Lexeme, Form, and Sense): https://phabricator.wikimedia.org/T181253 * Good progress on making usage tracking/RC integration more performant: https://phabricator.wikimedia.org/T113468 * Investigate on the slow problems encountered on Wikidata: https://phabricator.wikimedia.org/T182322 * Data Types library has been integrated to Wikibase: https://phabricator.wikimedia.org/T180454 * Repeating the callout from 2 weeks ago: https://phabricator.wikimedia.org/T181130 == German Technical Wishlist == * Blocked by: * Blocking: * Updates: == SoS Meeting Bookkeeping == * Updates: ** Sending out a survey to determine whether the meeting time is ideal or should be changed.