Good day everyone,
We've seen a problem on a self-hosted docker-compose based installation
regarding the blazegraph or wdqs index and the effect that deletion of
items, especially mass-deletion, has on it.
Now, I would have preferred to have a deeper understanding of this
before posting to the mailing list, but in this case, the wdqs service
really is like a black box to me and on top of that, I think that we're
in a situation where, regardless of the understanding of the underlying
cause, it won't change the current factual situation.
In a nutshell, what we observe is that at times, and I believe it might
be related to mass deletion ("nuking") of items, deleted items will not
be removed from the blazegraph index.
The main issue with that, besides simply returning false results, is
that some tools use both a SPARQL query, followed by an API request to
manipulate data, e.g. wikibaseintegrator.
With items remaining in the blazegraph index, but no longer existing on
mediawiki, this of course results in situations such as
{'name': 'wikibase-validator-no-such-entity', 'parameters':
['[[Item:Q342|Q342]]']…
Here, the SPARQL endpoint returned this item, but on the mediawiki
instance, it has long been deleted, resulting in an API error.
I unfortunately do not have logs of the requests made to the mediawiki
service by the wdqs-updater service back when this mass-deletion
happened, so I cannot tell if yes or no, all necessary requests were
made. I have traced the steps involved in deleting a single items and
seeing it being successfully deleted from the index though:
1. User: POST title=Item:Q3&action=delete
2. wdqs-updater -> mediawiki: api.php?action=query&list=recentchanges
3. mediawiki -> wdqs-updater: { type:log, title:Item:Q3, revid:0, …}
4. wdqs-updater -> mediawiki: Special:EntityData/Q3.ttl
I don't have full logs for the other case where the index does not get
updated correctly, but I do have for instance a log line like
18:31:27.750 [main] INFO o.w.q.r.t.change.RecentChangesPoller - Got 1
changes, from Q22@20220630163125|6522 to Q22@20220630163125|6522
And querying the SPARQL endpoint at 18:40 the Q22 item would still be
returned.
The logs from the wdqs container either to not contain any information
relevant to this problem or, alternatively, are so verbose that I'm
drowning in a sea of messages, being unable to understand what is going
on and even whether there is anything relevant to my problem or not.
My questions are these:
1. Is a problem like this known?
2. Is there any way to manually go into the blazegraph index and delete
that one record that no longer exists in mediawiki or to make
blazegraph purge it by somehow replaying the recentchanges entry, or
3. Will we need to drop the wdqs volume and recreate the entire index
from scratch with a sufficiently large $wgRCMaxAge value?
Thank you,
David Raison
--
*TenTwentyFour S.à r.l.*
www.tentwentyfour.lu <https://www.tentwentyfour.lu>
*T*: +352 20 211 1024
*F*: +352 20 211 1023
1 place de l'Hôtel de Ville
4138 Esch-sur-Alzette
*(apologies for cross-posting)*
Hi everyone,
The next Wikidata+Wikibase office hours
<https://www.wikidata.org/wiki/Category:Office_hour_notes> will take place
on Wednesday, July 27th 2022 at 16:00 UTC (18:00 Berlin time) in the Wikidata
Telegram group <https://t.me/joinchat/IeCRo0j5Uag1qR4Tk8Ftsg>.
*The Wikidata and Wikibase office hours are online events where the
development team presents what we have been working on over the past
quarter, and the community is welcome to ask questions and discuss
important issues related to the development of Wikidata and Wikibase.*
I am looking forward to seeing you all.
--
Mohammed Sadat
*Community Communications Manager for Wikidata/Wikibase*
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0) 30 577 116 2466
https://wikimedia.de
Keep up to date! Current news and exciting stories about Wikimedia,
Wikipedia and Free Knowledge in our newsletter (in German): Subscribe now
<https://www.wikimedia.de/newsletter/>.
Imagine a world in which every single human being can freely share in the
sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de
Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
*(apologies for cross-posting)*
Hello!
As we move into the summer months, I would like to let you know that
response times from Léa and me to requests and queries may be a bit longer
due to vacations and our current workload.
We are currently in the process of hiring a new team member to help with
Wikibase, and we hope to have them on board by October. In the meantime, we
ask for your patience and understanding.
Thanks,
--
Mohammed Sadat
*Community Communications Manager for Wikidata/Wikibase*
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0) 30 577 116 2466
https://wikimedia.de
Keep up to date! Current news and exciting stories about Wikimedia,
Wikipedia and Free Knowledge in our newsletter (in German): Subscribe now
<https://www.wikimedia.de/newsletter/>.
Imagine a world in which every single human being can freely share in the
sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de
Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
*[apologies for cross-posting]*
Hi everyone,
The software development team at Wikimedia Deutschland has open job
positions that I'd like to bring your attention to:
- Junior Product Manager Wikidata
<https://wikimedia-deutschland.softgarden.io/job/19886514?utm_campaign=googl…>
- Product Manager Wikibase Suite
<https://wikimedia-deutschland.softgarden.io/job/19887130/Product-Manager-Wi…>
Please review the job description and apply if you're interested or share
it with colleagues and friends.
Best,
--
Mohammed Sadat
*Community Communications Manager for Wikidata/Wikibase*
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0) 30 577 116 2466
https://wikimedia.de
Keep up to date! Current news and exciting stories about Wikimedia,
Wikipedia and Free Knowledge in our newsletter (in German): Subscribe now
<https://www.wikimedia.de/newsletter/>.
Imagine a world in which every single human being can freely share in the
sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de
Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
Hi,
Is it possible to change the URL/URI structure of a Wikibase. I found this
tutorial
<https://addshore.com/2019/11/changing-the-concept-uri-of-an-existing-wikiba…>,
but since I am not using Docker, I am struggling to follow it (also my
technical knowledge is rather limited).
Building on this question, is it possible to have two instances(options?)
for $wgServer on MediaWiki LocalSettings.php file? I am working with a
MediaWiki + Wikibase installation that encompasses both a wikipedia-style
part linked to a wikibase part. Right now, the articles and entities paths
are now the same, which is obviously not ideal.
Article URL structure: http://mydomain.pt/wiki/Name_of_the_person
Entity URL: http://mydomain.pt/wiki/Item:Q1234
For reference, the project uses MediaWiki 1.35.
Thank you very much in advance,
Rute Correia
*Wikimedian in Residence @ NOVA FCSH*
[image: Wikimedia Portugal]
<https://pt.wikimedia.org/wiki/Wikimedia_Portugal> [image: WP20Symbols
MediaWiki.svg] [image: Wikipedia20 Knowledge.svg]
[image: https://www.fcsh.unl.pt/] <https://www.fcsh.unl.pt/>