https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-11-23
= 2016-11-23 =
== Technology ==
=== Analytics ===
* Blockers: none
* on track for quaterly goals
* main project about edit data (mediawiki edit history reconstruction)
progressing,
* we are now calculating standard edit metrics for all wikis since the
beginning of time using denormalized edit history:
https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Denormalized_edit_h…
experimental dataset on pivot: https://goo.gl/XAMVsh
* working on productionizing infrasructure for event streams
* waiting for hardware for pageview API
* owning now statsv together with ops (utility that can consume kafka data
and report to graphana)
* Thanking discovery for contributing to our metric reporting tools
Upcoming:
Start design work to revamp information architecture of
http://stats.wikimedia.org
=== Performance ===
Not blocking, not blocked
* thanks everyone who attended the active/active DC meeting after I flagged
it here, it has helped getting the ball rolling on two blockers
* hidden tabs confirmed as messing with timing data, now excluded from perf
metrics
* investigating little-known legacy features in mediawiki thumbnailing to
decide whether we continue supporting them on Thumbor (302 redirects)
* second view tests added for firefox and IE in WebPageTest (was previously
only looking at Chrome)
* still active on thumbnail URL/API RFC discussion
* briefly discussed witth multimedia team setting up Thumbor for them to
leverage in their ImageTweaks extension
=== Security ===
* Security Reviews
* Linter review complete
* LoginNotify schedule for this week
* Continuing work on wiki account compromise remediation (T150554)
* Assistance needed -- e-mail to engineering@ is forthcoming with request
=== Services ===
* Blockers: none
* Updates:
** PDF render service deployed in codfw, eqiad and public exposure next
week
** New version of service-template-node: ES6 and ESLint are coming
=== Technical Operations ===
* '''Blockers'''
** IOS native app
*** Requesting timeline for Wikipedia iOS app requesting 0px thumbs:
https://phabricator.wikimedia.org/T147648https://phabricator.wikimedia.org/T151078
iOS 5.3.0 was shipped last week
** Performance ?
*** MW fix to return 400 on 0px thumbs
https://phabricator.wikimedia.org/T147784
* '''Blocking'''
** None
* Updates
** jobqueue woes https://phabricator.wikimedia.org/T151196
** kubernetes/calico work ongoing, goal on track
** dropping varnish 3 compatibility code from our puppet repos
** labsdb goals on track as well
=== Release Engineering ===
* Blocking
**
* Blocked
**
* Updates
** Mediawiki 1.28 tarball release this week!
== Product ==
=== Reading ===
==== Mobile Content Service (MCS) ====
* Board: https://phabricator.wikimedia.org/tag/mobile-content-service/
* Added announcements feed endpoint (public now). More info and request URL
at
https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/RESTBase_services_for_ap…
:
==== Android native app ====
* Last week:
** Continuing Q2 goals for Wikidata descriptions
** New fundraising announcement explore feed card in progress
** Now building against Android Nougat 7.1 API 25
** Fixing login and editing issues
** Lots of unit tests
* Next week (https://phabricator.wikimedia.org/project/view/2352/):
** More Q2 goals for Wikidata descriptions (tutorial and polish)
==== iOS Native App ====
* Last Week:
** Shipped 5.3.0 (In the news notifications & feed content, MCS backed
feed, language variant support, other bug fixes and enhancements)
https://phabricator.wikimedia.org/project/view/2220/
** Added announcement card to the feed (for user research and fundraising)
** Started update of data layer to fix issue with data access &
modification from widgets and notifications
** Started dynamic font size updates to the app
* This week: https://phabricator.wikimedia.org/project/view/2357/
** Finishing data layer update
** Continuing dynamic font size updates
** Other minor bug fixes for 5.3.1
==== Web ====
* Current sprint:
https://phabricator.wikimedia.org/tag/reading-web-sprint-86-%F0%9F%94%AA%F0…
** Stopping HoverCards A/B tests from Russian and Italian wikis
** New readers work
** Make PageImages return the image in the lead section
** MobileFrontend tech debt
** Trending service
** Hovercards rewrite
* Next week: probably the same stuff as the current week.
==== Reading Infrastructure ====
* not blocking
* Still looking for reviews on the API error/warning i18n patches
** https://gerrit.wikimedia.org/r/#/c/321402/ - Improve handling of Message
objects as Message parameters
** https://gerrit.wikimedia.org/r/#/c/321404/ - Add Message::listParam()
** https://gerrit.wikimedia.org/r/#/c/321405/ - Fix MediaTransformError
message handling
** *https://gerrit.wikimedia.org/r/#/c/321406/*
<https://gerrit.wikimedia.org/r/#/c/321406/> - API: i18n for warnings and
errors
** The extension patches depended on by that change are next in importance.
These are for OAuth, TitleBlacklist, GlobalBlocking, Translate, and
ConfirmEdit.
** All other WMF-deployed extensions affected by this change have patches
too, see *https://gerrit.wikimedia.org/r/#/q/topic:api-error-i18n/T47843*
<https://gerrit.wikimedia.org/r/#/q/topic:api-error-i18n/T47843>.
Non-WMF-deployed extensions are (mostly) not touched at this time, the
worst that should happen to them is wfDeprecated warnings eventually.
* https://gerrit.wikimedia.org/r/#/c/312865/ is blocked on review by
Security
=== Community Tech ===
* Not blocking
* Blocker: Need a security review for
https://phabricator.wikimedia.org/T150832 to proceed with exposing a couple
of table views on tool labs db
* Community Wishlist survey proposal phase over.
https://meta.wikimedia.org/wiki/2016_Community_Wishlist_Survey
* Did bug-fixing for Copypatrol (plagiarism detection tool) and launched
French version: https://tools.wmflabs.org/copypatrol/fr
* Ongoing RfC about changing default collation on Meta:
https://meta.wikimedia.org/wiki/Requests_for_comment/Switch_default_categor…
* Ongoing RfC about abandoned labs tools takeover:
https://meta.wikimedia.org/wiki/Requests_for_comment/Abandoned_Labs_tools
* Ongoing work with programs dashboard
=== Discovery ===
* BM25 scoring enabled on 10 larges wikis
* Discovery mission & roadmap presentation:
https://docs.google.com/presentation/d/1ctlqdLA__0OxDuO7mJEIDLP-xt9a7E4jv4I…
* Load-testing crosswiki searching backend code
* Portal updates per-language article count stats, dewiki joins the
lucrative 2M+ club :)
=== Editing ===
==== Language ====
* Blocked: T150512: WikiBase Repo tests failing with UsageException
** This is making it difficult to merge Translate patches. Issue seems to
be in database clearing in tests. QA/RelEng?
==== Collaboration ====
* Blocked: None
* Blocking: None
* Updates
** No deployments this week. Ongoing work on:
*** Mobile support for left nav of Special:Notifications
*** RecentChanges filters and filter framework for Edit Review Improvements
=== Fundraising Tech ===
* Big English fundraiser starts next week!
** repeating Greg's emailed plea:
https://lists.wikimedia.org/pipermail/engineering/2016-November/000331.html
** Stuff that could impact the fundraiser: GeoIP, ResourceLoader,
MessageCache, EventLogging, Hive webrequest tables
* CentralNotice: reviewing Aaron Schultz's latest MessageCache patch:
https://gerrit.wikimedia.org/r/#/c/318489
** We want to understand it really well before we deploy anything that
could affect banners
** If anyone with deep knowledge of MessageCache (Aaron, Gilles?) has time
for a quick video chat, Andrew Green has a few questions
** As always, more scrutiny and comments are welcome.
* Deployed Nirzar's mobile CSS fixes, looking great so far
* Minor caching optimizations for CiviCRM jobs
Hello everyone,
I have proposed a 2017 Dev Summit session titled "Improved editability,
tooling, reasoning, and performance by adopting DOM-based semantics for
wikitext" [1].
The TL:DR; summary is to treat a wikitext article as made up of document
fragments that are composed together instead of a concatenation of
strings which may (but often does not) yield well-formed HTML markup.
Rather than repeat what is already there on the task, I'll mention a
couple of benefits if you are an editor:
1. If you are a template editor, and you edit a template, it would be
much more efficient to update all affected articles. For the common use
case, you would simply replace the old output in an article with the new
output without having to reparse the article.
2. It is possible to think of edit tooling that lets you edit the
document at much finer granularities than sections without stepping on
each other's toes.
There are more details in a draft note [2]. Please don't be alarmed by
the "wikitext 2.0" term there. That is an umbrella term that we've been
throwing around in the Parsing Team for a bunch of related but different
proposals related to wikitext syntax and semantics. This phab task and
the wiki page is one proposal that concerns itself primarily with
semantics of wikitext markup and less about syntax. The only syntactic
feature considered is the heredoc-style syntax for some transclusions
[3] which would immensely benefit this approach.
Please comment on the task with your thoughts, ideas, concerns,
approval, criticism, or award tokens or subscribe to indicate interest. :-)
Thanks,
Subbu.
[1] https://phabricator.wikimedia.org/T149282
[2] https://www.mediawiki.org/wiki/Parsing/Notes/Wikitext_2.0
[3] https://phabricator.wikimedia.org/T114432
FYI to those who use these hosts during SWAT deploys.
----- Forwarded message from Giuseppe Lavagetto <glavagetto(a)wikimedia.org> -----
> Date: Tue, 22 Nov 2016 14:53:50 +0100
> From: Giuseppe Lavagetto <glavagetto(a)wikimedia.org>
> To: Operations Engineers <ops(a)lists.wikimedia.org>
> Subject: [Ops] mw1017, mw1099 are gone
>
> Hi all,
>
> due to an unfortunate chain of events, mw1017 and mw1099, the MediaWiki
> test appservers in eqiad, had to be decommissioned this morning. Their
> substitutes are mwdebug1001 and mwdebug1002
>
> So, not to break the deployers workflow, what is happening is what follows:
>
> - when you select mw1017 from the browser extension, the request will go do
> mwdebug1001
> - when you select mw1099, the request will go to mwdebug1002
>
> the extensions will be updated as soon as possible (in a way that will
> allow us an easy change of the server list without any extension release),
> and I am in the process of upgrading the documentation on wikitech.
>
> I apologize for the inconvenience,
>
> Giuseppe
> --
> Giuseppe Lavagetto, Ph.d.
> Senior Technical Operations Engineer, Wikimedia Foundation
> _______________________________________________
> Ops mailing list
> Ops(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/ops
----- End forwarded message -----
--
| Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E |
| Release Team Manager A18D 1138 8E47 FAC8 1C7D |
Hi,
a friend of mine is looking for examples of Bash code to edit MediaWiki via
API (as far as I understood, login, logout, get page text, put text to the
page (preferrably with custom tag); nothing more (unless needed)) for his
project of sharing some open data.
He has some other Bash scripts which generate some text which he needs to
save on wiki. Such text can be provided to the API edit part either via
pipe, as a variable content or saved in file (that's the preferred order of
possibilities).
If you have or know about any examples, please let me know (feel free to
reply off-list with attachments), so I can forward it to him.
Thanks for any help in advance.
Kind regards
Danny B.
Today it came to my attention, via https://phabricator.wikimedia.org/T151229,
that Phabricator's inbound email has been failing silently for several days.
Since no error messages were being delivered it easily went unnoticed over
the weekend.
I just fixed the root cause, however, an unknown number of replies to
Phabricator tasks have gone into the void without being recorded on the
intended tasks.
If you regularly interact with phabricator via email, I would like to ask
you to check your sent-mail to be sure your comments made it into
phabricator. You will likely need to re-send any messages that occurred
since at least Friday the 18th.
I apologize for the inconvenience. As a user of Phabricator's built-in
notifications I never even notice the email functionality, and as a result,
it went untested for too long.
Regretfully,
Mukunda Modell
Maintainer: Wikimedia Phabricator
Hello all,
Over the past few months the Design team members at the Wikimedia
Foundation (user experience [UX] designers, design researchers, user
experience engineers, and communications) have been working with Arthur
Richards from the Team Practices Group to identify the high-level themes
that motivate design at the WMF. These themes have been turned into a brief
statement of purpose, whose intent is to articulate the vision and purpose
behind design at the WMF. This statement will influence the future
direction of design work.
At this point the stakeholders are ready for a review of the draft
statement. The purpose of this review is to gather a common understanding
of its purpose, and to identify any key themes that may be missing from the
high-level discussion. On the wiki page for the statement, you'll find
these themes and what they encompass in the "Background" section. If you
have an observation, comment, or concern about what is listed there, please
bring it up on the talk page. If it is relevant to the review and
understanding of the statement, it will be looked at for future drafts. If
there are comments about design and the design process in general, we'll
hold on to those until a time when they can be addressed for the broader
discussion of design in general.
All that said, here are the links:
* <https://www.mediawiki.org/wiki/Design/Statement_of_purpose>
* <https://www.mediawiki.org/wiki/Talk:Design/Statement_of_purpose>
We look forward to seeing you on the wiki.
--
Keegan Peterzell
Technical Collaboration Specialist
Wikimedia Foundation
Since Friday, we've had a slow but steady stream of admin account
compromises on WMF projects. The hacker group OurMine has taken credit
for these compromises.
We're fairly sure now that their mode of operation involves searching
for target admins in previous user/password dumps published by other
hackers, such as the 2013 Adobe hack. They're not doing an online
brute force attack against WMF. For each target, they try one or two
passwords, and if those don't work, they go on to the next target.
Their success rate is maybe 10%.
When they compromise an account, they usually do a main page
defacement or similar, get blocked, and then move on to the next target.
Today, they compromised the account of a www.mediawiki.org admin, did
a main page defacement there, and then (presumably) used the same
password to log in to Gerrit. They took a screenshot, sent it to us,
but took no other action.
So, I don't think they are truly malicious -- I think they are doing
it for fun, fame, perhaps also for their stated goal of bringing
attention to poor password security.
Indications are that they are familiarising themselves with MediaWiki
and with our community. They probably plan on continuing to do this
for some time.
We're doing what we can to slow them down, but admins and other users
with privileged access also need to take some responsibility for the
security of their accounts. Specifically:
* If you're an admin, please enable two-factor authentication.
<https://meta.wikimedia.org/wiki/H:2FA>
* Please change your password, if you haven't already changed it in
the last week. Use a new password that is not used on any other site.
* Please do not share passwords across different WMF services, for
example, between the wikis and Gerrit.
(Cross-posted to wikitech-l and wikimedia-l, please copy/link
elsewhere as appropriate.)
-- Tim Starling
Hi,
as a followup to previous activities and ideas how to improve code
review, we've proposed a session at the upcoming Developer Summit to
concentrate on organizational and social aspects.
Sharing your experience and thoughts is welcomed:
https://phabricator.wikimedia.org/T149639
Thanks in advance,
andre
--
Andre Klapper | Wikimedia Bugwrangler
http://blogs.gnome.org/aklapper/
A few weeks ago our Executive Director gave a talk on "Privacy and
Harassment on the Internet" at MozFest 2016 in London. I encourage you to
read the transcript:
https://en.wikisource.org/wiki/Privacy_and_Harassment_on_the_Internet
Katherine argued that the Wikimedia project can take a lead role in
creating a culture of respect and inclusion online. I whole-heartedly
agree, and I hope you all do too. She concluded with:
"We have a lot of work to do. I know that. We know that. As Molly’s story
> illustrates, we are not there yet."
I'd like to open a broader discussion on how we get "there": how to
build/maintain places where we can get work done and control abuse and
vandalism while still remaining wide open to the universe of differing
viewpoints present in our projects. We can't afford to create filter
bubbles, but we must be able to provide users safe(r) spaces to work.
By habit I would propose that this be a technical discussion, on specific
tools or features that our platform is currently missing to facilitate
healthy discussions. But the "filter bubble" is a social problem, not a
technical one. Our project isn't just a collection of code; it's a
community, a set of norms and habits, and a reflection of the social
process of collaboration. A graph algorithm might be able to identify a
filter bubble and good UX can make countervailing opinions no more than a
click away, but it takes human will to seek out uncomfortable truth.
So although my endgame is specific engineering tasks, we need to start with
a broader conversation about our work as social creatures. How do we work
in the projects, how do we communicate among ourselves, and how do we
balance openness and the pursuit of truth with the fight against abuse,
harassment, and bias.
Let's discuss discussions!
Here are some jumping off points; feel free to contribute your own:
We currently use a mixture of Talk pages, Echo, mailing lists, IRC,
Phabricator, OTRS, Slack, Conpherence, and Google Doc on our projects, with
different logging, publication, privacy/identity, and other
characteristics. I tried to start cataloging them here:
https://lists.wikimedia.org/pipermail/wikimedia-l/2016-November/085542.html
Because of this diversity, we lack a unified code of conduct or mechanism
to report/combat harassment and vandalism.
Matt Flaschen replied in the above thread with an update on the Code of
Conduct for technical spaces:
https://lists.wikimedia.org/pipermail/wikimedia-l/2016-November/085542.html
...which should definitely help! The creation of a centralized reporting
mechanism, in particular, would be most welcome.
I created a proposal for the Wikimedia Developer Summit in January
discussing "safe spaces" on our projects:
https://phabricator.wikimedia.org/T149665
Subscribe/comment/click "award token" to support its inclusion in the dev
summit or to start a conversation there.
I have another, broader, proposal as well, on the "future of chat" on our
projects:
https://phabricator.wikimedia.org/T149661
Subscribe/comment/click "award token" there if that angle piques your
interest.
It seems that "groups of users" arise repeatedly as an architectural
meta-concept, whether it's a group of collaborators you want to invite to
an editing session, a group of users you want to block or ban, a group of
users who belong to a particular wikiproject, or who watch a certain page.
We don't really have a first-class representation of that concept in our
code right now. In previous conversations I've heard that people "don't
want <their wiki project> to turn into another facebook" and so have pushed
back strongly on the idea of "friend lists" (one type of group of users) --
but inverting the concept to allow WikiProjects to maintain a list of
"members of the wikiproject" is more palatable, more focused on the editing
task. From a computer science perspective "friend list" and "member of a
wikiproject" might seem identical--they are both lists of users--but from a
social perspective the connotations and focus are significantly different.
But who administers that list of users?
Perhaps we can build a system which avoids grappling with user groups
entirely. It was suggested that we might use an ORES-like system to
automatically suggest collaborators on an editing project based on some
criteria (like editing history), rather than force you or the WikiProject
to maintain an explicit list. Perhaps you can implement block lists to
combat harassment based entirely on keywords, not users. Do we trust the
machine to be more fair and less abusive than us mere mortals? Additional
ideas welcome! (I don't have a dedicated phab task for this, but
https://phabricator.wikimedia.org/T149665 might be appropriate if you want
to contribute off-list.)
Hopefully this has been enough to prime the pump.
Let's discuss discussions.
Let's live up to the hope placed in us by the Washington Post:
https://www.washingtonpost.com/news/wonk/wp/2016/10/25/somethings-terribly-…
Let's retake the lead on building and renewing a healthy collaborative
community. We can't afford to be complacent or content with the status
quo. Let's come up with new ideas, build them, find the problems, and try
again. It starts with deciding that we can do better.
--scott
--
(http://cscott.net)