Hello
*GitLab will be down this coming Monday, 27 February, from 10:00 AM–noon
UTC* for the datacenter switchover[0][1].
Much of this switchover window will be downtime, during which you will be
unable to access GitLab through ssh or the web interface.
Apologies for any inconvenience :(
The process will involve a full backup of the GitLab server in the eqiad
datacenter, which serviceops will then restore on the GitLab server in the
codfw datacenter.
You can follow along on IRC in the libera.chat #wikimedia-gitlab channel[2].
Tyler Cipriani (he/him)
Engineering Manager, Release Engineering
Wikimedia Foundation
[0]: <https://phabricator.wikimedia.org/T327920>
[1]: <https://phabricator.wikimedia.org/T329931>
[2]: <ircs://irc.libera.chat:6697/wikimedia-gitlab>
If your code doesn't read or write to databases in MediaWiki, feel free to
ignore this email.
Hello,
We have made changes to MediaWiki's Rdbms library that simplify and improve
how to get database connections and how to perform UPDATE queries. We
encourage you to adopt these and provide feedback here or via Phabricator.
== 1. Get a database connection ==
What changed:
To get a connection to the primary database, you previously needed the
following:
$lbFactory = MediaWikiServices::getInstance()->getDBLoadBalancerFactory();
$lbFactory->getMainLB()->getConnection( DB_PRIMARY );
And for a replica connection:
$lbFactory->getMainLB()->getConnection( DB_REPLICA );
You may now adopt the simpler $lbFactory->getPrimaryDatabase() and
$lbFactory->getReplicaDatabase().
In other words, instead of needing knowledge of the internal LoadBalancer,
its getConnection methods, and the DB_ constants, you only call
getPrimaryDatabase() or getReplicaDatabase().
We won’t deprecate LoadBalancer::getConnection() any time soon. If your
code depends on other LoadBalancer methods, feel free to keep your code
as-is. This change is aimed at the most common use case where you only need
a database connection.
The optional parameters have gotten more usable as well.
getPrimaryDatabase() only takes $domain, which means you no longer need to
skip over the non-applicable $groups parameter. getReplicaDatabase() takes
a string for $group (instead of an array in LB::getConnection).
If you maintain service classes that inject a LoadBalancer object, we
recommend transitioning to inject and type against LBFactory instead. This
removes exposure to LB altogether in most cases.
Currently, getPrimaryDatabase/getReplicaDatabase do not yet support
connection to external clusters (LB::getExternalLB). This is coming soon
(see below).
Reasoning:
We have been reducing complexity around LB/LBF for a while (T326274
<https://phabricator.wikimedia.org/T326274>). The distinction between these
is arbitrary in practice, each wears too many hats, and are not well
encapsulated internally.
Most importantly, LB is mostly internal to the Rdbms library. Most
developers only need a connection to perform read or write queries. The
added coupling can lead to mistakes, misunderstandings, and can feel
overwhelming. The LB documentation includes jargon aimed at Rdbms
maintainer, such as references to internal methods like
isReadyForRoundOperations, setTempTablesOnlyMode, getAnyOpenConnection,
appendShutdownCPIndexAsQuery, etc.
The getConnection() signature is fragile and has led to major outages due
to $domain being set incorrectly. E.g. if you need to connect to another
wiki, you have to set $domain in *both* getMainLB() and getConnection(),
otherwise *fireworks in production*. We keep this internally now to prevent
future outages.
Lastly, this change makes it possible to add a narrow typehint dedicated to
read-only connections on replica databases, separate from primary database
connections (more below).
Example adoption:
-
https://gerrit.wikimedia.org/r/c/mediawiki/core/+/891357
-
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DiscussionTools/+/890…
Adaption is being tracked in T330641
<https://phabricator.wikimedia.org/T330641>
What is next:
getPrimaryDatabase() and getReplicaDatabase() don't support "external"
databases yet (e.g. x1 at WMF
<https://wikitech.wikimedia.org/wiki/MariaDB#Extension_storage>). This is
coming soon. Details and discussion at T330590
<https://phabricator.wikimedia.org/T330590>.
In addition to reducing service dependencies to the ILBFactory typehint, we
are working on an even narrower interface (that ILBFactory extends) called
IConnectionProvider
<https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/refs/heads/…>
which only has four methods. It’s marked @internal for now, but that will
change after we hammer out some details.
== 2. IReadableDatabase typehint for replica connections ==
What changed:
When you typehint a parameter or class member to IDatabase where a replica
database is expected (DB_REPLICA), you may change it to the narrower
IReadableDatabas interface.
LBFactory::getReplicaDatabase returns IReadableDatabase.
Reasoning:
MediaWiki developers have at times performed write queries on a replica
connection by mistake, which reached production. While we have effective
protections against data corruption, our only option in production is to
throw a fatal error, which leads to an outage. With this change we move
detection to the very start of the development cycle, verified fully by
static analysis in CI, regardless of test coverage!
The new interface also helps you to self-document in a standard way whether
your service class only reads or also writes to the database. And in IDEs
we will no longer autocomplete write methods on what are known to be
replica connection objects. It’ll also make the coupling between the rest
of MediaWiki and extension with the rdbms library looser
<https://en.wikipedia.org/wiki/Loose_coupling#In_programming>.
Note that the IDatabase::query() is not part of IReadableDatabase. This
method is discouraged as it executes raw SQL that may include write
queries. If you call this method, continue to typehint IDatabase until you
transition to safe methods like select() or SelectQueryBuilder.
Example adoption:
-
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GrowthExperiments/+/8…
== 3. Introducing UpdateQueryBuilder ==
What changed:
In 2020, we introduced a chainable SelectQueryBuilder
<https://www.mediawiki.org/wiki/Manual:Database_access#SelectQueryBuilder>
for SELECT queries. To complement this, we now added a class called
UpdateQueryBuilder, for UPDATE queries. The following existing code uses
update() with unnamed positional parameters:
$db->update(
'image',
[ 'img_name' => 'foo' ],
[ 'img_name' => 'bar' ],
__METHOD__
);
You may now use the fluent style like so:
$db->newUpdateQueryBuilder()
->update( 'image' )
->set( [ 'img_name' => 'foo' ] )
->where( [ 'img_name' => 'bar' ])
->caller( __METHOD__ )
->execute();
Reasoning:
Similar to the SelectQueryBuilder, the fluent style is more readable. It
simplifies building of conditions, is less prone to mistakes, and is more
aligned with widely-used frameworks such as Doctrine (PHP) and Jooq (Java).
We have had at least two data corruption incidents in production due to
passing the wrong order of $set and $where condition (swapping them by
mistake). This reduces the chance of such data corruptions happening, which
alone was worth developing the builder for, before we get to the improved
ergonomics.
Example adoption:
-
https://gerrit.wikimedia.org/r/c/mediawiki/core/+/889627/5/includes/Categor…
Adaption is being tracked in T330640
<https://phabricator.wikimedia.org/T330640>.
What is next:
We will continue to develop query builders for DELETE and INSERT queries.
The work on migrating calls from ::select() to SelectQueryBuilder is
on-going (T311866 <https://phabricator.wikimedia.org/T311866>).
Thank you
--
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>
I would like to announce the availability of MediaWiki 1.39.2.
This release primarily is to fix various issues around database upgrades
that users have reported on both MySQL/MariaDB and PostgreSQL.
In the case of MySQL/MariaDB, there may have been the chance of data loss
(as always, please backup before upgrading!), and page views may have
resulted in an error like "The revision #0 of the page named "x" does not
exist." This was seen only when upgrading in a bigger jump between versions
(for example 1.31 to 1.39). More information can be seen in
https://phabricator.wikimedia.org/T326071.
https://phabricator.wikimedia.org/T328169 has been filed to write a
maintenance script to fix the issues caused by these upgrades. It is
expected this will be included in a later point release.
Various patches aimed at support for PHP 8.0, 8.1, and 8.2 have been
back-ported. This should fix some log spam, and MediaWiki should work fully
on versions PHP 8.0 and 8.1.
Reports of bugs with PHP 8.0, 8.1, or 8.2 support are particularly welcome,
and fixes will be back-ported when possible. Please see
https://phabricator.wikimedia.org/tag/php_8.0_support/,
https://phabricator.wikimedia.org/tag/php_8.1_support/ and
https://phabricator.wikimedia.org/tag/php_8.2_support/ for the relevant
work boards.
The tarballs have already been uploaded as of this email; the git tag has
also already been pushed.
== Release notes ==
Full release notes for 1.39.2:
https://phabricator.wikimedia.org/diffusion/MW/browse/REL1_39/RELEASE-NOTES…https://www.mediawiki.org/wiki/Release_notes/1.39
== Changes since MediaWiki 1.39.1 ==
* Localisation updates.
* (T325872) ChangeTags: Remove table name from condition.
* (T324895) MWCallbackStream: Add explicit $stream property.
* (T297031, T326039) PostgresUpdater: Move setDefault ahead of
changeNullableField.
* (T321319) Produce HTML for invalid JSON.
* (T215466, T326071) MigrateActors: Write to revision table (Follow-up
24115a8).
* (T223027) ReservedUsernames config: Add reserved names from maintenance
scripts.
* (T325000, T324896, T307631) Updated OOUI from v0.44.3 to v0.44.5.
* Remove /images .htaccess rules that are no longer relevant.
* Disable php in .htaccess of images directory as a hardening measure.
* (T322583) Include missing message parameter in message.
* LocalFileTest: use encodeBlob/decodeBlob for img_metadata.
* DatabaseSqlite: fix null blobs.
* rdbms: avoid pg_escape_bytea() call-style deprecation notices.
* (T322278) Improve LocalisationCache post-merge validation check.
* (T324408, T326367) Updated wikimedia/remex-html from 3.0.2 to 3.0.3.
* (T322278) Fix the remaining Phan failures on PHP 8.1.
* (T322278, T326367) Respond to some messages from Phan on PHP 8.1.
* Fix phan error when Excimer is enabled.
* (T326021) Add matrix: to $wgUrlProtocols.
* (T314099) stream wrapper: Declare $context class property.
* (T314099) libs\jsminplus: Declare JSNode::$expression.
* (T314096) composer.json: Updated composer/spdx-licenses from 1.5.6 to
1.5.7.
* (T326472) Upgrading cssjanus/cssjanus (v2.1.0 => v2.1.1).
* (T308536) rdbms: Remove deprecation mark for $wgSharedDB.
* (T215466, T326071) installer: Split drop action out of the SQL patch for
actor migration.
* (T322603) SqliteMaintenance.php: Fix fatally broken instanceof check.
* (T326377) rdbms: Use DBConnRef in SelectQueryBuilder.
* api/en.json: api-help-datatype-expiry add missing 'may'.
* (T317329) OutputPage: Fix undefined ['host'] in ImagePreconnect code.
* (T328222) Pass empty string to strlen() if schema is null for
PostgresDatabase.
* (T289926) SpecialRevisionDelete: Set default of '' for wpReason.
* (T155582, T328503) Fix XML dumps for content types with non-string
getNativeData().
* (T326886) PoolCounterRedis: Fix wrong cast, locks weren't being released.
* (T314099) revisiondelete: Replace dynamic property Status::$itemStatuses
* (T327821) skin: Restore default 'value' attribute in makeSearchButton().
* (T329198) ParamValidator: Improve paramvalidator-help-multi-max message.
* (T329415) Clear the statsd data buffer regardless of StatsdServer config.
* (T292348) WikiImporter: do not fail if upload entry in dump lacks 'text'
tag.
* (T330049) UnregisteredLocalFile: Don't call MimeAnalyzer if no path.
* (T324894 TempFSFile: Use a WeakMap for reference tracking if available.
* (T295637) Add no to fallback chain of nb and nn.
**********************************************************************
Download:
https://releases.wikimedia.org/mediawiki/1.39/mediawiki-1.39.2.tar.gzhttps://releases.wikimedia.org/mediawiki/1.39/mediawiki-1.39.2.zip
Download without bundled extensions:
https://releases.wikimedia.org/mediawiki/1.39/mediawiki-core-1.39.2.tar.gzhttps://releases.wikimedia.org/mediawiki/1.39/mediawiki-core-1.39.2.zip
Patch to previous version (1.39.1):
https://releases.wikimedia.org/mediawiki/1.39/mediawiki-1.39.2.patch.gzhttps://releases.wikimedia.org/mediawiki/1.39/mediawiki-1.39.2.patch.zip
GPG signatures:
https://releases.wikimedia.org/mediawiki/1.39/mediawiki-core-1.39.2.tar.gz.…https://releases.wikimedia.org/mediawiki/1.39/mediawiki-core-1.39.2.zip.sighttps://releases.wikimedia.org/mediawiki/1.39/mediawiki-1.39.2.tar.gz.sighttps://releases.wikimedia.org/mediawiki/1.39/mediawiki-1.39.2.zip.sighttps://releases.wikimedia.org/mediawiki/1.39/mediawiki-1.39.2.patch.gz.sighttps://releases.wikimedia.org/mediawiki/1.39/mediawiki-1.39.2.patch.zip.sig
Public keys:
https://www.mediawiki.org/keys/keys.html
Hi,
On Tue, Mar 1, 2016 at 3:36 PM, David Strine <dstrine(a)wikimedia.org> wrote:
> We will be holding this brownbag in 25 minutes. The Bluejeans link has
> changed:
>
> https://bluejeans.com/396234560
I'm not familiar with bluejeans and maybe have missed a transition
because I wasn't paying enough attention. is this some kind of
experiment? have all meetings transitioned to this service?
anyway, my immediate question at the moment is how do you join without
sharing your microphone and camera?
am I correct thinking that this is an entirely proprietary stack
that's neither gratis nor libre and has no on-premise (not cloud)
hosting option? are we paying for this?
-Jeremy
Hello everyone,
TL;DR if you are not deploying services to the codfw wikikube kubernetes
cluster, you can safely skip this.
Long version:
We will reinitialize the codfw wikikube kubernetes cluster with kubernetes
version 1.23 on 2023-02-21 09:00-16:00 UTC [1] (the actual process is expected
take a couple of hours within this window).
The date was chosen for convenience as we will have depooled all active/active
services from codfw for row B switch maintenance [2] anyways. As all traffic
will be drained beforehand we expect no user visible impact. However, for the
duration of the process, the kubernetes cluster will be unavailable to
deployers and thus efforts to deploy to it will fail or worse, not have the
expected outcomes.
This is normal until SRE serviceops announces that the cluster is fully
operational again.
SRE serviceops will be deploying all services before marking the cluster as
usable and pooling traffic back to it, so there will be no need for deployers to
re-deploy their services (apart from those already informed).
[1] https://phabricator.wikimedia.org/T329664
[2] https://phabricator.wikimedia.org/T327991
Regars,
Janis Meybohm
Hello everyone,
If you are interested in organizing or joining a hackathon event, but
cannot attend the in-person Hackathon event in May in Athens, Greece, this
email is for you!
We encourage communities, user groups or chapters to organize satellite
events connected to the in-person Hackathon. These events are to be
organized autonomously and share the hackathon's purpose: bringing the
global technical community together to connect, hack, run technical
discussions, and explore new ideas.
You can work with your wiki community to organize these events before,
during, or after the main event to onboard newcomers to the technical
aspects of the Wikimedia movement, hosting watch parties or meetups in your
region to offer an alternative to people who cannot join the in-person
event in Athens.
To obtain help with organizing an event, you can apply for funds via the *Rapid
Grants* maintained by the Wikimedia Foundation. The deadline to apply for
funding is *March 20*. When preparing for your event, you can reach out to
the Hackathon organizing team for support with resources, designing the
program, and guidance on getting involved in the global event.
Learn more about the satellite events, funding process, and a checklist for
organizing on the wiki page: <
https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2023/Satellite_events>
[1]
Cheers,
Srishti
On behalf of the Hackathon organizing team
[1] https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2023/Satellite_events
*Srishti Sethi*
Senior Developer Advocate
Wikimedia Foundation <https://wikimediafoundation.org/>
I ran across another reference to our lovely, important book creation tools
of yore, on a global Open Education forum
<https://connect.oeglobal.org/t/what-happened-to-wikipedia-book-creator/4341…>
.
What is the best place to discuss the broad, important version of this?
* How we support conversion to standalone formats, for reading and printing
* Clarity for Dirk (M2L <https://mediawiki2latex.wmflabs.org/>),
PediaPress, the PDF-feature
<https://m.mediawiki.org/wiki/Reading/Web/PDF_Functionality>team, and other
maintainers
This is important to a wide range of reuse, and the uncertainty around what
is possible, what is planned, and what is desired by various audiences
makes it hard to make progress even when it's obvious that is important.