On Mon, Sep 21, 2015 at 1:22 PM, Ryan Lane <rlane32(a)gmail.com> wrote:
> I know someone is working on an auth framework update, so I'm sure
> there'll be some changes necessary for that too.
>
We're planning on making the changes necessary for AuthManager in
WMF-deployed extensions (including LdapAuthentication and CentralAuth) as
part of the AuthManager project, but not any other bugs or requests.
We'll also look at non-WMF-deployed extensions, but we may not actually
make the changes in those cases.
More details on what exactly needs changing in extensions will be announced
to this mailing list when we're finished determining exactly what those
changes will be. But as a preview, some of the changes coming are:
- AuthPlugin is going away in favor of multiple co-existing
authentication providers.
- Real support for authentication methods other than "username and
password", instead of hacking around the login form.
- Support for pluggable pre-authentication steps (e.g. throttles,
captcha) without hooking into the login form.
- Support for pluggable post-authentication steps (e.g. forcing a
password change, second-factor auth) without a mess of hooks like
AbortLogin and AbortNewAccount.
- Support for other methods of tying the request to an authenticated
session, no more UserLoadFromSession hook.
See also the original RFC
<https://www.mediawiki.org/wiki/Requests_for_comment/AuthManager>, and
T89459 <https://phabricator.wikimedia.org/T89459> and its many subtasks,
and Gerrit change 195297 <https://gerrit.wikimedia.org/r/#/c/195297/> for
the work-in-progress.
--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
I haven't been actively maintaining the LDAP extension for MediaWiki for
over two years. There's not really much that needs to change, but some
basic love and care is likely a good idea. The LDAP extension is one of the
most popular MediaWiki extensions, so it wouldn't be great if it was broken
for long periods of time.
If anyone would like to take this extension over, please feel free.
- Ryan
Hi,
We have scheduled an upgrade of mailman (https://lists.wikimedia.org) for:
Wednesday, September 9, 2015 at 2:00:00 PM UTC ( 7:00 AM PDT, 16:00 CEST)
The scheduled mainteance window is 4 hours (or less).
During this time please expect all the mailing lists (web interface
and email) to be down.
We will shut down the old server, then sync all pending mail to the
new server and bring it back up there.
The upgrade includes mailman from 2.1.13 to 2.1.18 and the server OS
will become a Debian jessie server and will be virtualized.
More things we expect to be fixed by this (excerpts):
"DMARC improvements solve issues users have with Yahoo and Outlook
regarding mail."
"Password reminder link is now on the roster page (private archives)."
"Emails can be automatically accepted to private lists if a poster
password is used"
"Cookies now set the secure flag when over HTTPS"
"Emails are validated more accurately. Spam prevention."
"Sessions can be invalidated through logouts on administrator and
moderation interfaces."
refs: https://wikitech.wikimedia.org/wiki/Mailman ,
https://phabricator.wikimedia.org/T105756 ,
Will follow-up with another mail once it's done.
--
Daniel Zahn <dzahn(a)wikimedia.org>
Operations Engineer
I just now see that there was a report published about last year's security
breach on Freenode. The report, written by NCC Group’s Cyber Defence
Operations team, may be of interest to infrastructure security and tech ops
people:
https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2014/octob…
Pine
Freenode occasionally purges its databases of "expired nicks, channels and
accounts". If you have a nick, channel or account that you want to keep and
haven't used recently, now is a good time to do so. See
https://blog.freenode.net/2015/09/services-database-purge/ for more info.
Please send any questions to Freenode staff.
Pine
The Discovery Department has launched an experimental tile and static maps
service available at https://maps.wikimedia.org.
Using this service you can browse and embed map tiles into your own tools
using OpenStreetMap data. Currently, we handle traffic from *.wmflabs .org
and *.wikivoyage .org (referrer header must be either missing or set to
these values) but we would like to open it up to Wikipedia traffic if we
see enough use. Our hope is that this service fits the needs of the
numerous maps developers and tool authors who have asked for a WMF hosted
tile service with an initial focus on WikiVoyage.
We'd love for you to try our new service, experiment writing tools using
our tiles, and giving us feedback <https://www.mediawiki.org/wiki/Talk:Maps> .
If you've built a tool using OpenStreetMap-based imagery then using our
service is a simple drop-in replacement.
Getting started is as easy as
https://www.mediawiki.org/wiki/Maps#Getting_Started
How can you help?
* Adapt your labs tool to use this service - for example, use Leaflet js
library and point it to https://maps.wikimedia.org
* File bugs in Phabricator
<https://phabricator.wikimedia.org/tag/discovery-maps-sprint/>
* Provide us feedback to help guide future features
<https://www.mediawiki.org/wiki/Talk:Maps>
* Improve our map style <https://github.com/kartotherian/osm-bright.tm2>
* Improve our data extraction
<https://github.com/kartotherian/osm-bright.tm2source>
Based on usage and your feedback, the Discovery team
<https://www.mediawiki.org/wiki/Discovery> will decide how to proceed.
We could add more data sources (both vector and raster), work on additional
services such as static maps or geosearch, work on supporting all
languages, switch to client-side WebGL rendering, etc. Please help us
decide what is most important.
https://www.mediawiki.org/wiki/Maps has more about the project and related
Maps work.
== In Depth ==
Tiles are served from https://maps.wikimedia.org, but can only be accessed
from any subdomains of *.wmflabs .org and *.wikivoyage.org. Kartotherian
can produce tiles as images (png), and as raw vector data (PBF Mapbox
format or json):
.../{source}/{zoom}/{x}/{y}[(a){scale}x].{format}
Additionally, Kartotherian can produce snapshot (static) images of any
location, scaling, and zoom level with
.../{source},{zoom},{lat},{lon},{width}x{height}[(a){scale}x].{format}.
For example, to get an image centered at 42,-3.14, at zoom level 4, size
800x600, use https://maps.wikimedia.org/img/osm-intl,4,42,-3.14,800x600.png
(copy/paste the link, or else it might not work due to referrer
restriction).
Do note that the static feature is highly experimental right now.
We would like to thank WMF Ops (especially Alex Kosiaris, Brandon Black,
and Jaime Crespo), services team, OSM community and engineers, and the
Mapnik and Mapbox teams. The project would not have completed so fast
without you.
Thank You
--tomasz
In the next RFC meeting, we will discuss the following RFC:
* Improving extension management
<https://www.mediawiki.org/wiki/Requests_for_comment/Improving_extension_man…>
The meeting will be on the IRC channel #wikimedia-office on
chat.freenode.net at the following time:
* UTC: Wednesday 21:00
* US PDT: Wednesday 14:00
* Europe CEST: Wednesday 23:00
* Australia AEST: Thursday 07:00
-- Tim Starling
Mediawiki maintenance scripts have moved. The old "misc/maintenance.pp"
that had all cronjobs and scripts in one huge file is gone.
Now you find all the maintenance scripts in separate files, one per job,
and inside the mediawiki module.
So if you look for anything please see
./puppet/modules/mediawiki/manifests/maintenance now.
Each job has its own .pp file now, which should make it easier to use and
changes easier to review.
It also got rid of another large file in the manifests/misc/ structure
which we want to remove entirely and contains it with other mediawiki setup.
refs: https://gerrit.wikimedia.org/r/#/c/178873/https://phabricator.wikimedia.org/T88597http://puppet-compiler.wmflabs.org/891/terbium.eqiad.wmnet/
--
Daniel Zahn <dzahn(a)wikimedia.org>
Operations Engineer
FYI
---------- Forwarded message ----------
From: Yuvi Panda <yuvipanda(a)gmail.com>
Date: Wed, Sep 16, 2015 at 6:25 PM
Subject: [Tools] Kubernetes picked to provide alternative to GridEngine
To: labs-announce(a)lists.wikimedia.org
We have picked kubernetes.io to provide an alternative to GridEngine
on Tool Labs! See https://phabricator.wikimedia.org/T106475 for more
details about the evaluation itself,
https://docs.google.com/spreadsheets/d/1YkVsd8Y5wBn9fvwVQmp9Sf8K9DZCqmyJ-ew…
for the spreadsheet with the evaluation matrix and
https://lists.wikimedia.org/pipermail/labs-l/2015-August/003955.html
for the previous email listing pros and cons of the others solutions
that we have considered.
== Rough Timeline ==
* October: Start beta testing by opening up Kubernetes to whitelisted
tools. Allows people to run arbitrary Docker images in Kubernetes,
both for continuous jobs and web services. If you are interested in
this,please add a comment to https://phabricator.wikimedia.org/T112824
* December: Work on migrational tooling to assist in switching from
GridEngine to Kubernetes should be in a good place. This will begin
with a '--provider=kubernetes' parameter to the webservice command
that will allow people to easily switch to Kubernetes for webservice.
We will have something similar implemented for jsub / jstart.
* January: Kubernetes cluster is opened up for all tools users. Fairly
complete switching between GridEngine and Kubernetes for at least
continuous jobs and webservices.
== What does this give developers? ==
# A software stack that is made up of widely-used and
actively-developed software components. We are looking to dramatically
reduce the surface of code that we write and maintain in-house.
# (When out of beta) More stability and reliability.
# It allows us to get rid of a lot of our customizations (the entire
webservice setup, for example) which has proven to be a lot of work
and flaky at times.
# We can migrate tools that don't require NFS off of it, and since it
has historically been one of our flakiest servies this allows tools to
opt-out of it and get a lot more reliability.
# Using Docker allows more user freedom in a structured way that is
easier to support.
# Has an active upstream / is used elsewhere as well (Google Container
Engine, Microsoft Azure, RedHat OpenShift, AWS, etc.), so much more
help / community available when users run into issues than we have now
# Probably more :) It opens up a lot of possibilities and I'm sure
developers will use it createively :)
== Do I need to change anything? ==
No, especially if you do not want to. We think Docker and Kubernetes
are exciting technologies, so we would like volunteers who are
interested in exploring these platforms to have the option of getting
their hands dirty. At the same time, it is important for us to avoid
complicating life for people for whom the current setup works well. If
we are successful, the migration to Docker and Kubernetes will be
seamless, and users of Tool Labs will not have to know what either
Docker or Kubernetes are, let alone how to operate them.
We will begin by offering arbitrary Docker image execution only.
Eventually we will make it super-easy to switch between the current
setup and Kubernetes, and allow people to be able to use this without
having to know what Docker is or what Kubernetes is. That will slowly
be built over the next few months.
Absolutely nothing changes for developers or users right now.
== Will GridEngine be going away? ==
Not anytime soon!
Thanks!
--
Yuvi Panda T
http://yuvi.in/blog
--
Yuvi Panda T
http://yuvi.in/blog
Are there any stable APIs for an application to get a parse tree in
machine-readable format, manipulate it and send the result back without
touching HTML?
I'm sorry if this question doesn't make any sense.
