(If you don’t work with pagelinks table, feel free to ignore this message)
Hello,
Here is an update and reminder on the previous announcement
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
regarding normalization of links tables that was sent around a year ago.
As part of that work, soon the pl_namespace and pl_title columns of
pagelinks table will be dropped and you will need to use pl_target_id
joining with the linktarget table instead. This is basically identical to
the templatelinks normalization that happened a year ago.
Currently, MediaWiki writes to both data schemes of pagelinks for new rows
in all wikis except English Wikipedia and Wikimedia Commons (we will start
writing to these two wikis next week). We have started to backfill the data
with the new schema but it will take weeks to finish in large wikis.
So if you query this table directly or your tools do, You will need to
update them accordingly. I will write a reminder before dropping the old
columns once the data has been fully backfilled.
You can keep track of the general long-term work in T300222
<https://phabricator.wikimedia.org/T300222> and the specific work for
pagelinks in T299947 <https://phabricator.wikimedia.org/T299947>. You can
also read more on the reasoning in T222224
<https://phabricator.wikimedia.org/T222224> or the previous announcement
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
.
Thank you,
--
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>
Hello,
We have communicated
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
this change in August 2022 but here is a reminder that if you query
externallinks table in wikireplicas, you will need to rework your queries.
Changes are: el_to, el_index and el_index_60 fields will be dropped and you
need to query el_to_domain_index and el_to_path fields instead.
The data has been migrated in all wikis except English Wikipedia, Wikimedia
Commons and Wikidata, you can keep track of the data migration in T326314
<https://phabricator.wikimedia.org/T326314>.
In late June, we will stop updating the old fields and will start dropping
them
This is done to optimize storage of external links (drastically reducing
its size by removing duplication) and enabling user requested features such
as resolving two <https://phabricator.wikimedia.org/T14810> fifteen-year-old
tickets <https://phabricator.wikimedia.org/T17218> or easier aggregating of
external links domain [1], something that wasn’t possible until now.
[1] For example, If you want to get list of the top-linked websites, you
can query `SELECT el_to_domain_index, count(*) from externallinks group by
el_to_domain_index order by count(*) desc limit 50;`
Thank you and sorry for the inconvenience.
--
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>
Hello all,
The first language community meeting is coming up in a few weeks - November
17th, 16:00 UTC.
If you're interested, you can sign up on this wiki page: <
https://www.mediawiki.org/w/index.php?title=Wikimedia_Language_engineering/…>
[1].
We would like to invite all of you who are involved in creating content or
managing technical aspects across different language communities. This will
be a participant-driven meeting, where we collectively discuss specific
technical issues related to our language wikis and work together to find
possible solutions. This could involve anything from fixing a broken
template on the Kurdish wiki to brainstorming ideas for growing content on
the Tulu Wiktionary, currently in the Wikimedia Incubator, or celebrating
the creation of Fon Wikipedia, to using MinT for content translation.
You can view the meeting structure and add ideas for discussion to the
notes document here: <
https://etherpad.wikimedia.org/p/language-community-meeting-november-2023>
[2]
If you need interpretation support from English to another language, please
let us know by sending an email to ssethi(a)wikimedia.org.
Looking forward to your participation!
Cheers,
Srishti & Jon
[1]
https://www.mediawiki.org/w/index.php?title=Wikimedia_Language_engineering/…
[2]
https://etherpad.wikimedia.org/p/language-community-meeting-november-2023
*Srishti Sethi*
Senior Developer Advocate
Wikimedia Foundation <https://wikimediafoundation.org/>
Hi,
Toolforge's Harbor instance will briefly be down for a version upgrade from
2.5 to 2.9 this Wednesday at 8:00 UTC.
https://phabricator.wikimedia.org/T346241
<https://phabricator.wikimedia.org/T346241>
This should not affect any tools that are not using the new build service,
nor any tools that are already running.
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service
If you are using the builds service, you will not be able to run any new
builds, or start a job or a webservice from an image built with the
build service while Harbor is down.
We will send an update before starting maintenance work, and once
everything is back up and running.
--
Slavina Stefanova (she/her)
Software Engineer | Developer Experience
Wikimedia Foundation
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
Hello,
The Toolforge admin team is happy to announce that the Toolforge Build
Service[0] is now available in open beta.
The Build Service is intended to allow more tools to migrate off the Grid
Engine and to make the process for deploying code to
Toolforge easier and more flexible, by building container images with the
specific dependencies for each tool.
Here are quick highlights of some of the current key features:
1. Build your tool from source code, using you language's dependency
management tool, no dockerfiles, no scripts, no manual steps
2. Use industry-wide standards[1] no vendor lock-in by using upstream
buildpacks
3. Support for many languages out of the box[2]
4. Envvars - Create and manage environment variables and secrets that are
available at runtime.[3]
5. Ability to install packages from the Ubuntu repositories[4]
6. Improved resiliency and resource usage by allowing NFS-less
webservices[5], if you don't need NFS
7. Test your image locally, or anywhere[6]
Please review the current known limitations here[7]
We also have a growing list of tutorials for various languages[8]
During this open beta, we invite you to actively participate and share your
feedback replying to this thread or through irc, and if you
find any issues or have any feature suggestions, you can use this task
template[9].
Your insights will help us enhance and tailor the Build Service to meet the
needs of your tools.
The plan is to have this phase run for the next months, and if no big
issues are found, promote it to global availability phase 1 (GA1)
while we work on adding automatic triggering and deployment, for which we
will do a second round of beta testing for those specific features.
This unblocks the last step to migrate out of the grid, so we request all
grid users to give it a try and report any issues they might find,
there's no big changes expected for the currently implemented features, so
any work done now will help later.
Thank you for being a part of this journey. We look forward to your
invaluable feedback and collaboration as we strive to provide a better
developer experience.
[0]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service
[1]: https://buildpacks.io/
[2]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Supported_…
[3]:https://wikitech.wikimedia.org/wiki/Help:Toolforge/Envvars_Service
[4]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Installing…
[5]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Using_NFS_…
[6]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Testing_lo…
[7]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Known_curr…
[8]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Tutorials_…
[9]: https://w.wiki/7kpi
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
Hi!
There will be a small network interruption next Monday at around 13:00 UTC
as we will be doing some cleanup on the Openstack after network
re-architecture (see https://phabricator.wikimedia.org/T348140).
It will affect all the CloudVPS and other services hosted there (including
toolforge, PAWS, quarry and superset). VMs traffic to the internet will be
cut for a short period, hopefully for a few seconds, while internal traffic
will not be affected, but if you have any open ssh session to your VMs or
login.toolforge.org, it might timeout or get dropped, and any web access to
projects will not work during the downtime.
We will update on this email thread when we start and when we have finished.
This will help stabilize the network and avoid bigger outages in the
future, so thanks for your patience!
--
David Caro
SRE - Cloud Services
Wikimedia Foundation <https://wikimediafoundation.org/>
PGP Signature: 7180 83A2 AC8B 314F B4CE 1171 4071 C7E1 D262 69C3
"Imagine a world in which every single human being can freely share in the
sum of all knowledge. That's our commitment."
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
Hi,
We will be upgrading the Toolforge Kubernetes cluster cluster on October
18th starting at around 11:00 UTC.
The expected impact is that tools running on the Kubernetes cluster will
get restarted a couple of times over the course of the few hours it takes
for us to upgrade the entire cluster. This includes tools that use the jobs
framework and tools that run web services using the default Kubernetes
backend. The ability to manage tools will remain operational.
Taavi
--
Taavi Väänänen (he/him)
Site Reliability Engineer, Cloud Services
Wikimedia Foundation
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
I see T336057 was just closed. Looking at the docs <https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#View_web_service_logs>, I'm unclear how this works. The docs say ", the output from the webservice command is stored by the Toolforge Kubernetes infrastructure as long as the web service is running." So, what happens when a service exits (i.e. crashes)? Does that mean the logs for that service disappear?