Hello
*GitLab will be down this coming Tuesday, 02 May, from 09:00 - 12:00 UTC* for
the datacenter switchover[0].
Much of this switchover window will be downtime, during which you will be
unable to access GitLab through ssh or the web interface.
The process will involve DNS changes and a full backup of the GitLab server
in the codfw datacenter, which will then be restored on the GitLab server
in the eqiad datacenter.
You can follow along on IRC in the libera.chat #wikimedia-gitlab channel[1].
[0]: <https://phabricator.wikimedia.org/T335504>
[1]: <ircs://irc.libera.chat:6697/wikimedia-gitlab>
Hello all!
The Search Platform Team usually holds an open meeting on the first
Wednesday of each month. Come talk to us about anything related to
Wikimedia search, Wikidata Query Service (WDQS), Wikimedia Commons Query
Service (WCQS), etc.!
Feel free to add your items to the Etherpad Agenda for the next meeting.
Details for our next meeting:
Date: Wednesday, May 3, 2023
Time: 16:00-17:00 UTC / 08:00 PDT / 11:00 EDT / 17:00 CET
Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours
Google Meet link: https://meet.google.com/vgj-bbeb-uyi
Join by phone: https://tel.meet/vgj-bbeb-uyi?pin=8118110806927
Have fun and see you soon!
Guillaume
--
*Guillaume Lederrey* (he/him)
Engineering Manager
Wikimedia Foundation <https://wikimediafoundation.org/>
Hello,
I use the extension AutoCreatePage
(https://www.mediawiki.org/wiki/Extension:AutoCreatePage) in a project
at my University. But it does not work
in current mediawiki versions as it was last updated . So I try to adapt
it to work in mediawiki 1.39.3. It almost works now and I'm considering
releasing it but don't know how to proceed.
I forked the github project
(https://github.com/mkroetzsch/AutoCreatePage/pull/16) but got no
response for now. There are also older pull requests not answered.
Should I somehow replace the existing extension with my new one or would
it be better to make my fork a new extension with a different name? It
does not feel right, because I use
a lot of the code from the old extension.
I hope this is the correct mailing list for this topic. Otherwise I
would be thankful if you could direct me to the correct place to ask
about this.
Kind Regards,
Oliver Tautz
Hello,
We have two breaking changes to announce.
Breaking change to Rdbms Database subclasses
There are no Database subclasses currently known in Codesearch (besides
those built-in to core). But, since the class is documented as "stable to
extend" in API, an announcement is in order.
First, an explanation on what is happening. Currently, queries flow in the
Rdbms library as follows: When Database::select() or similar is called
(directly, or via the query builders), the Rdbms library internally builds
the SQL text and then calls the general-purpose function Database::query().
Subsequently, ::query() then runs several regexes over the SQL text to
figure out information about this query (which tables are affected, whether
it’s a write query, what the query “verb” is, and so on). This is taxing
and slow. The appservers at WMF, for example, spend 0.34% of CPU on web
requests just to extract information from already-formatted SQL queries for
logging purposes. This logic drastically reduces readability and
maintainability of Rdbms internals.
To fix this, we've internally introduced a new class called Query, to
represent the SQL text in a more structured way in the first place. This
avoids having to first format structure data into SQL text only to
reverse-engineer it right afterwards.
This change requires changing the protected Database::executeQuery() method
to no longer accept a string argument and instead require a Query object.
Additionally IDatabase::getTempTableWrites() has become a private method.
See the patch <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/910756> or the
ticket <https://phabricator.wikimedia.org/T326181> for more information.
The public Database::query() method continues to support a string argument,
but now also supports a Query object. When given a string, it falls back to
the old regexes to extract and create a Query object. With the newly
introduced UnionQueryBuilder, we believe there are no longer cases where
MediaWiki extensions have to call Database::query, and can instead adopt
query builders for all their queries. Direct calls to Database::query
outside of MediaWiki core are now highly discouraged.
Breaking change to typehints of IDatabase
Many parts of core typehint to IDatabase, but most times these always
return a replica database. For example, ApiBase::getDB(). This means the
interface technically allows calling insert(), and this is indeed valid,
but would also immediately throw a fatal error once the method is entered.
Going forward, most of these will be typehinted to IReadableDatabase
instead, which is a narrower interface (34 public methods instead of 83).
This is not a breaking change in practice, because calls to those methods
would already produce an error, but the typehint makes it official now. This
<https://gerrit.wikimedia.org/r/c/mediawiki/core/+/911269> is the patch
changing ApiBase::getDB(), and we expect more such changes to other parts
of MediaWiki over the coming months. Those will not be individually
announced.
Best
--
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>
Hello,
As part of ongoing refactors and improvements to the Rdbms component in
MediaWiki, we have made some changes that might affect your work. Read
the February
announcement
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
for previous changes in this area.
New DeleteQueryBuilder and UnionQueryBuilder
We introduced the chainable SelectQueryBuilder
<https://www.mediawiki.org/wiki/Manual:Database_access#SelectQueryBuilder>
in 2020 to create SELECT queries. This "fluent" API style allows each
segment of the query to be set through a discoverable method name with rich
documentation and parameter types. Last February we introduced
UpdateQueryBuilder in the same vein.
The new DeleteQueryBuilder similarly supersedes use of IDatabase::delete()
for building DELETE operations. Check change 913646
<https://gerrit.wikimedia.org/r/c/mediawiki/core/+/913646/1/includes/user/Us…>
for how this is used in practice.
Union queries are fairly rare in our codebases. Previously, one would have
to bypass the query builders (by calling the underlying SQL text formatters
like Database::selectSqlText) and pass several raw SQL strings through
Database::unionQueries and then to Database::query. As you can see, this is
not optimal. The new UnionQueryBuilder enables you to combine several
SelectQueryBuilder objects and then use the familiar fetchResultSet()
method on the resulting UnionQueryBuilder object. You can see an example of
this in change 906101
<https://gerrit.wikimedia.org/r/c/mediawiki/core/+/906101/5/maintenance/find…>
.
IConnectionProvider typehint
In February, we introduced the LBFactory->getPrimaryConnection() and
LBFactory->getReplicaConnection() methods to improve ergonomics around
getting database connections, and the IReadableDatabase typehint for
replica connections.
We're now introducing the IConnectionProvider typehint as a stable
interface holding the above methods without the rest of LBFactory. Meaning,
if you all you need is a connection, you can typehint to
IConnectionProvider. This is an extremely narrow interface (four public
methods!) compared to LBFactory (42 public methods). This will reduce
coupling and should ease testing as well. We already adopted
IConnectionProvider in several MediaWiki core components, which made a
notable dent in MediaWiki's PhpMetrics complexity score
<https://doc.wikimedia.org/mediawiki-core/master/phpmetrics/complexity.html>
.
This backwards-compatible change can be adopted through only a change in
typehint. MediaWiki’s LBFactory service implements IConnectionProvider.
Dependency injection remains unchanged.
You can see an example in change 913649
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DiscussionTools/+/913…>.
We recommend changing variable names like $lbFactory to $dbProvider for
consistency in usage.
Future changes
We will continue transforming MediaWiki core to adopt query builders,
IConnectionProvider, and IReadableDatabase. And, we'll continue helping
various core components and extensions to phase out their direct calls to
LoadBalancer::getConnection(). This should improve code health,
readability, testability, and overall coupling scores in MediaWiki.
Our next announcement will introduce InsertQueryBuilder, and we're
considering a query builder for "UPSERT" queries as well. Lastly, we are
planning to discourage remaining use of raw SQL in "where" conditions by
introducing an expression builder (T210206
<https://phabricator.wikimedia.org/T210206>).
You can help
Nearly every feature of MediaWiki requires retrieving or storing
information from a database at one point or another. Reworking these is a
long tail of effort. Any help in adopting query builders, and updating code
away from ILoadBalancer and ILBFactory would be really appreciated.
Together we can improve readability and code health (now with less
footguns!).
Until the next update,
--
Amir Sarabadani (he/him), Staff Database Architect
Timo Tijhof, Principal Performance Engineer
Hi Community Metrics team,
This is your automatic monthly Phabricator statistics mail.
Accounts created in (2023-04): 296
Active Maniphest users (any activity) in (2023-04): 1109
Task authors in (2023-04): 562
Users who have closed tasks in (2023-04): 306
Projects which had at least one task moved from one column to another on
their workboard in (2023-04): 283
Tasks created in (2023-04): 1921
Tasks closed in (2023-04): 1999
Open and stalled tasks in total: 52957
* Only open tasks in total: 51955
* Only stalled tasks in total: 1002
Median age in days of open tasks by priority:
Unbreak now: 13
Needs Triage: 844
High: 1117
Normal: 1845
Low: 2519
Lowest: 2700
(How long tasks have been open, not how long they have had that priority)
Active Differential users (any activity) in (2023-04): 5
To see the names of the most active task authors:
* Go to https://wikimedia.biterg.io/
* Choose "Phabricator > Overview" from the top bar
* Adjust the time frame in the upper right corner to your needs
* See the author names in the "Submitters" panel
TODO: Numbers which refer to closed tasks might not be correct, as
described in https://phabricator.wikimedia.org/T1003 .
Yours sincerely,
Fab Rick Aytor
(via community_metrics.sh on phab1004 at Mon 01 May 2023 12:00:22 AM UTC)