Hi all,
(Reposting https://www.mediawiki.org/wiki/Topic:Tyvfh19mba4pway9 here to
garner more input.)
I'm working on Extension:GlobalPreferences and trying to figure out how
best to do things with all preferences, after they've been defined (in
order to show various extra Preferences-form bits and pieces that depend
on knowing about all preferences). At the moment, we're using
$wgExtensionFunctions and hacking the $wgHooks global to add a new
callback at the end of $wgHooks['GetPreferences'].
One idea is to add a new MediaWiki service called 'PreferencesFactory',
that can be used to retrieve a new Preferences object. Extensions would
then be able to use the MediaWikiServices hook to redefine the
PreferencesFactory (with MediaWikiServices::redefineService()). Of
course, only one extension would be able to do that (which maybe is a
bit odd).
Apart from being able to override the Preferences class, a service for
this would also mean the Preferences class could be refactored
(gradually?) to not be such a collection of static methods.
The proposed patch is: https://gerrit.wikimedia.org/r/#/c/374451/
I'd love to hear anyone's ideas about this, including completely
different and better ways to do things. :-)
Another idea is to add a new hook, after GetPreferences. This wouldn't
be as flexible as the PreferencesFactory idea, but is a lot simpler.
Thanks,
Sam.
Hi all,
I'm an Outreachy <https://www.outreachy.org/alums/> intern and as a part of
the internship, I'm working on a project - WikiCV
<https://en.wikipedia.org/wiki/Wikipedia:WikiCV>.
Through this project we (my mentors - Gergő Tisza and Stephen LaPorte and
I) want to create a contribution summarizing tool which (unlike the
existing ones that focus on statistics and are hard to interpret for
someone not familiar with Wikipedia editing) highlights contributions in an
easy-to-understand manner.
I'm writing this to gather inputs from prospective users of this tool, that
is all the Wikipedia editors!
Basically I want to understand what all things would you like to see in a
tool like this? Or what all would you write in your Wikipedia CV?
You can give in your inputs through mail (I'd be more than happy to get one
:)) or fill out this form <https://goo.gl/forms/slmOreey5iVPWzFx1>. My work
is largely dependent on your inputs, so please pour in your comments/views.
Your help will be quite appreciated!
Eagerly waiting for your inputs :)
Thanks,
Megha Sharma
Hello from Analytics Team!
We are happy to announce the Alpha release of Wikistats 2. Wikistats has
been redesigned for architectural simplicity, faster data processing, and a
more dynamic and interactive user experience. First goal is to match the
numbers of the current system, and to provide the most important reports,
as decided by the Wikistats community (see survey) [1]. Over time, we will
continue to migrate reports and add new ones that you find useful. We can
also analyze the data in new and interesting ways, and look forward to
hearing your feedback and suggestions. [2]
You can go directly to Spanish Wikipedia
https://stats.wikimedia.org/v2/#/es.wikipedia.org
or browse all projects
https://stats.wikimedia.org/v2/#/all-projects
The new site comes with a whole new set of APIs, similar to our existing
Pageview API but with edit data. You can start using them today, they are
documented here:
https://wikitech.wikimedia.org/wiki/Analytics/AQS/Wikistats
FAQ:
Why is this an alpha?
There are features that we feel a full-fledged product should have that are
still missing, such as localization. The data-processing pipeline for the
new Wikistats has been rebuilt from scratch (it uses distributed-computing
tools such as Hadoop) and we want to see how it is used before calling it
final. Also while we aim to update data monthly, it will happen a few days
after the month rolls because of the amount of data to move and compute.
How about comparing data between two wikis?
You can do it with two tabs but we are aware this UI might not solve all
use cases for the most advanced Wikistats users. We aim to tackle those in
the future.
How do I file bugs?
Use the handy link in the footer:
https://phabricator.wikimedia.org/maniphest/task/edit/?title=Wikistats%20Bu…
How do I comment on design?
The consultation on design already happened but we are still watching the
talk page:
https://www.mediawiki.org/wiki/Wikistats_2.0_Design_Project/RequestforFeedb…
[1]
https://www.mediawiki.org/wiki/Analytics/Wikistats/DumpReports/Future_per_r…
[2] https://wikitech.wikimedia.org/wiki/Talk:Analytics/Systems/Wikistats
*https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-12-13
<https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-12-13>*
*= **2017-12-1**3** =*
== Callouts ==
* s8 database master switchover scheduled for 9th January
https://phabricator.wikimedia.org/T181645 [please keep in callouts until
Jan 9th]
* LDAP update for user thiemowmde needed:
https://phabricator.wikimedia.org/T181130
* Kafka jumbo cluster (eqiad) ready for testing connections via TLS. This
new version of kafka enables consumption via given timestamp - Discovery
will use it for wdqs. Aiming to switch some of our clients this week.
== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by:
* Blocking:
* Updates:
** Released 5.7.2 (high priority fix for
https://phabricator.wikimedia.org/T69015 ) (
https://phabricator.wikimedia.org/tag/ios-app-v5.7.2/ )
** Continuing work on 5.7.3 (Faster article loading, other minor
enhancements) for release before the end of the year (
https://phabricator.wikimedia.org/project/view/2913/ )
** Continuing work on 5.8 (Reading Lists) for release next year (
https://phabricator.wikimedia.org/project/view/3131/ )
==== Android native app ====
* Blocked by:
* Blocking:
* Updates:
* Released 2.7.221 to production
** fully configurable feed, On This Day cards in feed, new Randomizer,
new Black theme.
* Continuing to work on performance improvements with large numbers of
reading list pages.
==== Reading Web ====
* Blocked by:
* Blocking:
* Updates:
* Still working on chromium service. Almost finished performance testing.
* Re-enabling page previews A/B test to collect data to influence a
decision on performance.
==== Reading Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** Reading lists is in production
** will look at patrolling support for WP0 piracy next
** Media endpoint pretty getting ready to be exposed
** Finalizing touches for summary endpoint (in Beta Cluster) but may need
more adjustments for when to return 204 (No Content).
** Getting close for references endpoint
==== Multimedia ====
* Blocked by: N/A
* Blocking: N/A
* Updates
** 3D: A few changes to be made, on track to deploy sometime soon after the
holidays.
** MediaInfo: Prototyping and wireframing currently underway for first
features.
==== Discovery ====
* Blocked by:
* Blocking:
* Updates:
** lots of new portal automation documentation
https://gerrit.wikimedia.org/r/#/c/396407/
===== Maps =====
* Blocked by: N/A
* Blocking: N/A
* Updates:
** Frontend Mediawiki maps integration came in #1 in Community Wishlist:
https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey/Tracking
** Other: Paul Norman re-elected to OpenStreetMap Foundation board
=== Contributors ===
==== Editing ====
* Blocked by:
* Blocking:
* Updates:
==== Parsing ====
* Blocked by:
* Blocking:
* Updates:
==== Global Collaboration ====
* Blocked by: Security on re-review of
https://phabricator.wikimedia.org/T144467
* Blocking: Ops on Flow dumps still; Matt comes back on Monday and he's the
expert
* Updates:
** Nothing of note, mostly been fixing regressions recently
** Matt comes back on Monday (Dec 18)
==== Contributors Design ====
* Blocked by:
* Blocking:
* Updates:
==== UI Standardization ====
** No OOUI release this and upcoming weeks till January
* Ongoing:
** OOUI & based products:
*** 'constructive' flag in has been dropped entirely, please replace code
remainders with 'progressive'. Will go in effect in 1st release 2018.
*** icons: Work on icon set to be more harmonious and align to WikimediaUI
Style Guide https://phabricator.wikimedia.org/T177432 finishing up
*** Consider changing :hover tools and menu background to use Accent90.
https://phabricator.wikimedia.org/T166560
** Unify SVG markup across Foundation products
https://phabricator.wikimedia.org/T178867
** Continuous work and per-project SVGO based optimizations
=== Community Tech ===
* Wrapping up our survey, tightening loose ends from this year.
== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:
** Load testing druid prior to launch of wikistats 2.0 - It provides good
enough concurrentcy and response times. Last snapshot of data for
wikistats2 is of the quality we like. Aiming to announce launch this
Thursday: https://stats.wikimedia.org/v2/#/bm.wikipedia.org
** Working on new set of APIs to power map visualizations, pageviews per
country monthly
** Kafka jumbo cluster ready for testing connections via TLS, this new
version of kafka enables consumption via given timestamp. Discovery will
use it for wdqs. Aiming to switch some of our clients this week
** Notebook1002 got recommissined as a kafka host - Almost there (hard
drives issues)
** Superset is productionized - We are testing it internally in the teasm
and will anounce soon
=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
=== Fundraising Tech ===
* Blocked by: nothing
* Blocking: nothing
* Updates:
** Fundraiser going pretty smoothly (knock wood)
** Trawling logs for errors to fix
** Investigating potential banner impression shortfall for WMDE
** Getting some civi stuff merged upstream
** fixes & enhancements to internal dashboard
=== MediaWiki Platform ===
* Blocked by:
* Blocking:
* Updates:
** MediaWiki 1.30 Released
** code review:
*** https://gerrit.wikimedia.org/r/#/c/392990/ and
https://gerrit.wikimedia.org/r/#/c/392991/ for Global Collaboration
*** MCR patches
** Actor table patches are almost ready to merge
** script to clean up imported usernames ran on a limited set of wikis with
no errors so will be run on all wikis
** PSR-4 autoloading support merged
** namespacization and firejail work continues
** went through some bug backlogs including incident response, submitted
some small changes
** two meetings with representatives of Gamepedia (owned by AWS/
twitch.tv/curse)
** Tim: will go to TechCom meetings on Wednesday afternoon, then on
vacation after that until January 1
=== Performance ===
* Blocked by: n/a
* Blocking: n/a
* Updates:
** NavTiming extension updates should be going out quite soon. Don't
expect any impact on other teams.
** ChronoogyProtector caused problems last week, esp. for Wikidata. Fix is
in place, many thanks to the folks who jumped in to help out with that.
** mcrouter puppet config waiting on additional code review. May need some
additional review cycles from TechOps.
** ProxySQL SSL termination - may need more time from Jaime, not yet sure.
** Series of blog posts on Thumbor migration published to the main WMF blog
** WebPageReplay now working with Firefox.
=== Release Engineering ===
* Blocked by: n/a
** MW Platform team: https://phabricator.wikimedia.org/T182344 - "Blocker
must be a local user or a name that cannot be a local user"
* Blocking: n/a
** Blocking ORES with some scap issues.
* Updates:
** [MW Train] Reminder! This is your last week of deployments for the
year/quarter! No non-emergency deploys starts the week of December 18th.
** [Meta] Q3 goal planning
** [Security/Ruby] T180878 Upgrade RuboCop and Rubyzip (Ruby)
*** All done except for Minerva (needs a +2) and mediawiki/debian (unsure
if needed).
** [Security/Jenkins] Upgraded a bunch of Jenkins plugins Monday morning EU
time (after a bunch of security releases).
** [Phabricator] Exploring the use of Selenium tests for search quality
regressions
** [TechDebt] Getting further on the definition of “steward”. We will be
talking with Victoria and Toby “soon”.
** [SSD] One more merge (*https://gerrit.wikimedia.org/r/#/c/395570/*
<https://gerrit.wikimedia.org/r/#/c/395570/)>)
<https://gerrit.wikimedia.org/r/#/c/395570/)> and our quarterly goals are
complete.
** [PostMortem] ORES post-mortem completed last week:
*https://etherpad.wikimedia.org/p/Post-Mortem-T181006*
<https://etherpad.wikimedia.org/p/Post-Mortem-T181006>
=== Research ===
* Blocked by:
* Blocking:
* Updates:
=== Scoring Platform ===
* Blocked by:
** Blocked on releng, by scap issues. We're getting lots of help.
Unfortunately, this is affecting Q2 goal of ours (deploying to new ORES
cluster).
* Blocking:
* Updates:
** Reverted models about to go out for iswiki and eswikiquote
*** Need an ops merge for https://gerrit.wikimedia.org/r/#/c/398078/
*** git-lfs -- super excited. Seems like there's not much left before we
can use it.
=== Search Platform ===
* Blocked by: none
* Blocking: none
* Updates:
* New blog post about failed queries:
https://blog.wikimedia.org/2017/12/12/failed-queries-fear-of-missing-out/
* Fixes for completion suggester merged, to be deployed soon
https://phabricator.wikimedia.org/T178474
* Added tuning for wikidata prefix search
https://phabricator.wikimedia.org/T182136
* Porting to browser tests to Selenium mostly done, setting up CI runs
* Categories in WDQS are now auto-updated weekly from dumps (see
https://www.mediawiki.org/wiki/Wikidata_query_service/Categories )
* Working on improvements to LTR training
* Working on tuning fulltext search for Wikidata
=== Security ===
* Blocked by:
* Blocking:
* Updates:
* Security plugin for phan has been announced and published (yay!)
*
** Reviews:
*** Google MT
*** mediawiki-services-chromium-render
*** stacktraces on wikis
*** git mirroring to diffusion
=== Services ===
* Blocked by: none
* Blocking: none
* Updates:
** Parsoid storage for wikipedias moved to Cassandra 3. Please report
anything weird you notice with VE
** Parsoid and MCS content-type bumped
** HTMLCacheUpdate for all non-wikipedia projects processed on Kafka
=== Technical Operations ===
* Blocked by:
* Blocking:
* Updates:
** Asia DC (eqsin) on site installation complete
** Monitoring of production k8s cluster setup
** More work on porting system metrics to Prometheus
** HHVM on API appservers occasionally gets stuck - investigation in
https://phabricator.wikimedia.org/T182568
== Wikidata ==
* Good progress on opening the default Wikibase API for custom entity types
(namely Lexeme, Form, and Sense): https://phabricator.wikimedia.org/T181253
* Good progress on making usage tracking/RC integration more performant:
https://phabricator.wikimedia.org/T113468
* Investigate on the slow problems encountered on Wikidata:
https://phabricator.wikimedia.org/T182322
* Data Types library has been integrated to Wikibase:
https://phabricator.wikimedia.org/T180454
* Repeating the callout from 2 weeks ago:
https://phabricator.wikimedia.org/T181130
== German Technical Wishlist ==
* Blocked by:
* Blocking:
* Updates:
== SoS Meeting Bookkeeping ==
* Updates:
** Sending out a survey to determine whether the meeting time is ideal or
should be changed.
Hi,
You can now use a PSR-4 autoloader[1] in MediaWiki core and via
extension.json.
If you're not familiar with PSR-4, it's a standard which tells
autoloaders that if a class is named Foo\Bar\Baz, it can be found at
$basePath/Bar/Baz.php. This means that we don't need to keep a mapping
of classes to filenames, and instead just need to register the namespace
with the autoloader.
In extension.json, you can register PSR-4 compliant namespaces with the
"AutoloadNamespaces" property[2]. You can also look at the
ContentTranslation, Linter, or ORES extensions for examples.
[1] http://www.php-fig.org/psr/psr-4/
[2]
https://www.mediawiki.org/wiki/Manual:Extension.json/Schema#AutoloadNamespa…
-- Kunal / Legoktm
Hello everyone,
For the last little while I have been working on a new tool
to automatically detect common security issues in MediaWiki
extensions.
The tool can detect a number of issues, including:
* XSS
** We include here using wfMessage( 'foo' )->text()
when you should have used ->escaped() or ->parse().
* Sql injection
* Shell injection
* PHP deserialization vulnerabilities (A little buggy on this one)
In the future, it will likely also detect double escaping issues.
Of course, as with any static analysis tool, there will be instances
of false positives, as well as things it cannot detect.
I've now reached the stage where I feel the tool is useful,
and would really like people to test it out and give feedback.
Note: the tool has a requirement of php 7.0 (neither higher nor lower)
see https://www.mediawiki.org/wiki/Continuous_integration/Phan#Dependencies
for how to install php 7.0 if your system doesn't have it.
To test with your extension, simply do:
$ composer require --dev mediawiki/phan-taint-check-plugin
and then merge into the scripts directive of composer.json
"scripts": {
"seccheck": "seccheck-mwext",
"seccheck-fast": "seccheck-fast-mwext"
}
and simply run
composer seccheck
seccheck will take about 3 minutes and use lots of ram (~2 GB),
seccheck-fast won't test certain things involving hooks,
but will work in about 27 seconds and use much less ram.
This assumes that your extension is installed in the extensions/
subdirectory of MediaWiki.
In the future we may make this into a non-voting jenkins job.
If you are not making a MediaWiki extension, there is also
a "seccheck-generic" script you can use, which should work
with any PHP project. It is also possible to customize the script
for other projects that have custom escaping methods.
Generic mode is not well tested yet.
See the README for more information about the tool:
https://github.com/wikimedia/Phan-Taint-Check-Plugin/blob/master/README.md
Anyways, I hope this is useful, and am very eager to
hear feedback. I also hope that this will not only be useful
for Wikimedia, but also helpful to the third party extension
development community. Please test it and let me know what
you think.
Cross-posting.
---------- Forwarded message ----------
From: Danny Horn <dhorn(a)wikimedia.org>
Date: Thu, Dec 14, 2017 at 7:59 AM
Subject: [Wikimedia-l] Community Wishlist Survey: Top 10 wishes of 2017!
To: Wikimedia Mailing List <wikimedia-l(a)lists.wikimedia.org>
Hi everyone,
The Community Tech team is happy to announce the top 10 wishes from the 2017
Community Wishlist Survey!
More than 1,100 people participated in the survey this year -- proposing,
discussing and voting on 214 ideas. There was a two-week period in November
to submit and discuss proposals, followed by two weeks of support voting.
The top 10 proposals with the most support votes now become Community
Tech's backlog of projects to evaluate and address.
And here's the new top 10:
#1. Maps improvements (154 support votes)
#2. Ping users from the edit summary (127)
#3. Programs and events dashboard (111)
#4. Blame tool (110)
#5. Infobox wizard (106)
#6. Article Alerts for more languages (102)
#7. Auto-save edits (96)
#7. Thanks notification for log entries (tie, 96)
#9. SVG translation (94)
#10. Commons deletion notification bot (91)
You can see the whole list here, with links to proposals, project pages and
Phabricator tickets:
https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey/Results
So what happens next?
In 2018, the Community Tech team is responsible for investigating and
addressing the top 10 wishes. If there's a wish in the top 10 that we can't
work on, because it's unfeasible or because another group is working on it,
then we'll explain why we can't.
To get updates on our progress:
There are project pages for each of the top 10 wishes, which you can put on
your watchlist. We'll update them as the project progresses. (At time of
writing, these are just skeletons; actual information on each project is
still to come.) Feel free to post questions and suggestions on the project
talk pages: https://meta.wikimedia.org/wiki/Category:Community_Tech_-
_Current_projects
If you're familiar with the Phabricator ticketing system, the main Phab
task for each wish is noted on the Results page. You can also subscribe to
those tickets for updates.
We also publish several status reports through the year, to keep people
updated. You can watch the main Community Tech page for updates:
https://meta.wikimedia.org/wiki/Community_Tech
There are more questions and answers on the Wishlist Survey FAQ:
https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey/FAQ
Thanks to everybody who proposed, discussed, debated and voted on ideas in
this year's Wishlist Survey!
_______________________________________________
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/
wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/
wiki/Wikimedia-l
New messages to: Wikimedia-l(a)lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
<mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
--
Niharika
Software Engineer
Community Tech
Wikimedia Foundation
Hi all,
The experimental Trending Service[1] will be sunset on December 14th, 2017.
We initially deployed this service to evaluate some real time features in
the mobile apps centered on delivering more timely information to users.
After some research [2], we found that it did not perform well with users
in that use case.
At this point there are no further plans to integrate the service into our
products and so we are going to sunset the service to reduce the
maintenance burden for some of our teams.
We are going to do this more quickly than we would for a full stable
production API as the usage of the end point is extremely low and mostly
from our own internal projects. If you this adversely affects any of your
work or you have any other concerns, please let the myself or the Reading
Infrastructure team know.
Thanks to all the teams involved with developing, deploying, researching
and maintaining this service.
P.S. This service was based off of prototypes Jon Robson had developed for
detecting trending articles. He will be continuing his work in this area. I
encourage you to reach out to him if you were interested in this project.
[1] https://en.wikipedia.org/api/rest_v1/#!/Feed/trendingEdits
[2]
https://meta.wikimedia.org/wiki/Research:Comparing_most_read_and_trending_e…
--
Corey Floyd
Engineering Manager
Readers
Wikimedia Foundation
cfloyd(a)wikimedia.org