Alright, we are now ready to do this. New date: June 14 2018, around
14:00 UTC.
On Tue, Jun 5, 2018 at 9:31 AM, Andrew Otto <otto(a)wikimedia.org> wrote:
> Hi,
>
> We need to delay this switchover. We’ve made some changes to the plan
> that require a bit more prep work. Follow https://phabricator.
> wikimedia.org/T185225 for more details.
>
> I’ll reply with another announcement email when we set a new date.
>
> - Andrew Otto
> Senior Systems Engineer, WMF
>
>
>
> On Tue, May 15, 2018 at 11:43 AM, Andrew Otto <otto(a)wikimedia.org> wrote:
>
>> Hi all!
>>
>> *If you are not an active user of the EventStreams service, you can
>> ignore this email.*
>>
>> We’re in the process of upgrading
>> <https://phabricator.wikimedia.org/T152015> the backend infrastructure
>> that powers the EventStreams service. When we switch EventStreams to
>> the new infrastructure <https://phabricator.wikimedia.org/T185225>, the
>> ‘offsets’ AKA Last-Event-IDs will change.
>>
>> Connected EventStreams SSE clients will reconnect and not be able to
>> automatically consume from the exact position in the stream where they left
>> off. Instead, reconnecting clients will begin consuming from the latest
>> messages in the stream. This means that connected clients will likely miss
>> any messages that occurred during the reconnect period. Hopefully this
>> will be a very small number of messages, as your SSE client should
>> reconnect quickly.
>>
>> This switch is scheduled to happen on June 5 2018, at around 17:30 UTC.
>>
>> Let us know if you have any questions.
>>
>> Thanks!
>> - Andrew Otto
>> Senior Systems Engineer, WMF
>>
>>
>>
>>
>>
>>
>>
>>
>
Hi all!
*If you are not an active user of the EventStreams service, you can ignore
this email.*
We’re in the process of upgrading
<https://phabricator.wikimedia.org/T152015> the backend infrastructure that
powers the EventStreams service. When we switch EventStreams to the new
infrastructure <https://phabricator.wikimedia.org/T185225>, the ‘offsets’
AKA Last-Event-IDs will change.
Connected EventStreams SSE clients will reconnect and not be able to
automatically consume from the exact position in the stream where they left
off. Instead, reconnecting clients will begin consuming from the latest
messages in the stream. This means that connected clients will likely miss
any messages that occurred during the reconnect period. Hopefully this
will be a very small number of messages, as your SSE client should
reconnect quickly.
This switch is scheduled to happen on June 5 2018, at around 17:30 UTC.
Let us know if you have any questions.
Thanks!
- Andrew Otto
Senior Systems Engineer, WMF
Hello,
We are pleased to announce that service-runner v2.6.0 has been released.
The important update in this release is the ability to start workers
concurrently using the `startup_concurrency` config stanza~[1].
Its default value is 1, which keeps the start-up procedure the same as
pre-v.2.6.0. When the value is greater than 1, the start-up procedure is as
follows. The first worker is started alone. After the successful completion
of its start-up, service-runner starts the rest of the workers
(`num_workers - 1` of them) in batches of `startup_concurrency` workers
until all of them are up and running.
Doing so allows for a faster start-up process of your service and
consequently offers a more predictable behaviour of the service during
deploys/restarts. Note, however, that starting workers in batches incurs
the penalty of increased resource consumption during the start-up process.
Therefore, we recommend values between 2 and 4. In order to make use of
this feature simply update your dependency to ^2.6.0 and add the config
stanza to your production configuration file~[2].
On behalf of the Services team,
Marko Obrovac, PhD
Senior Services Engineer
Wikimedia Foundation
[1] https://github.com/wikimedia/service-runner#config-loading
[2] eg. https://gerrit.wikimedia.org/r/#/c/427466/
Hello,
We are rolling out page previews to enwiki next Tuesday~[1,2]. Since this
is a multi-hour effort, I have cancelled the usual services deployment
window, as we will be completing the final roll-out round at that time.
If there is something that you need to deploy Tuesday, please let me know
and we will find a suitable time for it. I apologise for the late notice
and inconvenience.
Cheers,
Marko Obrovac, PhD
Senior Services Engineer
Wikimedia Foundation
[1] https://phabricator.wikimedia.org/T191101
[2] https://wikitech.wikimedia.org/wiki/Deployments#Tuesday,_April_17
Heya,
I asked this on IRC but didn't get any replies so I'm following it up this
way.
I have a question about the newer metrics REST v1 API: is there a way to
specify how many top articles to pull from
https://wikimedia.org/api/rest_v1/#!/Pageviews_data/get_metrics_pageviews_t…
or is 1k hardcoded? Old metrics data was available that had the most viewed
pages but that disappeared with the change to the new API.
The reason I ask is because we (https://endlessos.com) are trying to
rebuild our stale encyclopedia apps for offline usage but are space-limited
and would only like to include the most likely pages that would be looked
at that can fit within a size envelope that varies with the device in
question (up to 100k article limit probably) but the new API doesn't
provide us with the tools to figure out the rankings cleanly (other than
rate-limiting on our side and checking every single article's metric
endpoint for counts).
So the main question is: do we have a way to get this data out with the
current API? If this data is not available, can the "metrics/pageviews/top"
API be augmented to maybe have a `skip` and/or `limit` params like other
similar services that have this type of filtering?
Thanks,
..........................................................................
Srdjan Grubor | +1.314.540.8328 | Endless <http://endlessm.com/>
FYI
---------- Forwarded message ----------
From: Daniel Zahn <dzahn(a)wikimedia.org>
Date: 27 March 2018 at 19:24
Subject: [Ops] new deployment server (tin -> deploy)
To: Operations Engineers <ops(a)lists.wikimedia.org>
Hi,
good old tin.eqiad.wmnet was out of warranty and running jessie,
so as part of our hardware refresh goal it had to be replaced by something
new.
We now have deploy1001.eqiad.wmnet and it's running on stretch with PHP7
and we just switched deployment servers and Mukunda is running the first
deploy from it as we speak.
<+logmsgbot> !log twentyafterfour@deploy1001 Started scap: Deploy
1.31.0-wmf.27 to test wikis
Here are the related puppet changes that switched it and added stretch
support:
https://gerrit.wikimedia.org/r/#/q/project:operations/
puppet+branch:production+topic:deploy1001
Additionally mwscript needed a way to detect php5 or php7, for that see:
https://gerrit.wikimedia.org/r/#/c/422348/
There are also more details on today's SAL and on:
https://phabricator.wikimedia.org/T175288
tin has not been killed yet and for now remains a scap master, just in case.
--
Daniel Zahn <dzahn(a)wikimedia.org>
Operations Engineer
_______________________________________________
Ops mailing list
Ops(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ops
On 06/03/2018 19:44, Marko Obrovac wrote:
> Hello,
>
> Today during the scheduled deploy window of moving JobQueue jobs from
> the Redis-based infrastructure to the new EventBus-based one, a
> combination of PEBKAC and Gerrit trying to be smart caused an accidental
> merge of the mw core's master branch into wmf.23. Luckily, the mistake
> was spotted before the wmf.23 branch got updated on tin, so the blast
> radius was limited to delaying the Euro-SWAT window by 30 minutes~[1].
> Big thanks to Antoine and Giuseppe for their help in discovering and
> resolving the issues.
>
> You can read the full incident report here~[2]. The TL;DR is that
> currently MW core repository is configured in such a way so as to allow
> Gerrit to do implicit (and silent!) merges, even for cherry-picked
> change-sets, which is something we might want to change in the near future.
>
> Cheers,
> Marko
>
> [1] Thanks, James F, for patiently waiting on the resolution!
> [2] https://wikitech.wikimedia.org/wiki/Incident_documentation/20180306-MasterM…
>
> Marko Obrovac, PhD
> Senior Services Engineer
> Wikimedia Foundation
Hello,
That Gerrit feature has been disabled by default for all repositories by
setting receive.rejectImplicitMerges = true in All-Projects.
https://gerrit.wikimedia.org/r/#/c/416704/
Implicit merging of a branch into another might be helpful sometime, but
most probably all typical use cases would be due to a mistake and could
lead to a catastrophe. I blame Gerrit default behavior on that one :]
I would like to thanks Marko and Petr to have followed the deployment guide:
* always check current workspace and remote before rebasing:
git remote update
git log HEAD..HEAD@{u}
* halt immediately and ring the bell in case of doubt. Surely have ~90
unwanted commits was a red alarm.
Thank you!
--
Antoine Musso
Dear all,
is there a plan which Node versions will be used in the WMF production
data centers in the following years:
https://github.com/nodejs/Release#release-schedule
In particular, I for my research projects I would like to develop only
for one particular node version to reduce the maintanence effort,
i.e., with regard to security patches.
Best
physikerwelt
--
Moritz Schubotz
Researcher
isg.uni-konstanz.de/people/moritz-schubotz
+49 1578 047 1397
One interesting thing that I noticed about the trending edits API is that
it was fairly useful in identifying articles that were under attack by
vandals or experiencing an edit war. A lot of times a vandal will just sit
on an article and keep reverting back to the vandalized version until an
admin shows up, which can sometimes take a while. If you tweak the
parameters passed to the API, you can almost get it to show nothing but
edit wars (high number of edits, low number of editors).
This makes me think that this API is actually useful, it's just targeted to
the wrong use case. If we built something similar, but that just looked for
high numbers of revert/undos (rather than edits), and combined it with
something like Jon Robson's trending edits user script (
https://en.wikipedia.org/wiki/User:Jdlrobson/Gadget-trending-edits.js), we
could create a really powerful tool for Wikipedia administrators to
identify problems without having to wait for them to be reported at AN/I or
AIV.
On Tue, Dec 12, 2017 at 7:25 AM, Corey Floyd <cfloyd(a)wikimedia.org> wrote:
> Just a reminder that this is happening this Thursday. Please update any
> tools you have before then. Thanks!
>
>
> On Fri, Dec 1, 2017 at 3:30 PM Corey Floyd <cfloyd(a)wikimedia.org> wrote:
>
> > Hi all,
> >
> > The experimental Trending Service[1] will be sunset on December 14th,
> 2017.
> >
> > We initially deployed this service to evaluate some real time features in
> > the mobile apps centered on delivering more timely information to users.
> > After some research [2], we found that it did not perform well with users
> > in that use case.
> >
> > At this point there are no further plans to integrate the service into
> our
> > products and so we are going to sunset the service to reduce the
> > maintenance burden for some of our teams.
> >
> > We are going to do this more quickly than we would for a full stable
> > production API as the usage of the end point is extremely low and mostly
> > from our own internal projects. If you this adversely affects any of your
> > work or you have any other concerns, please let the myself or the Reading
> > Infrastructure team know.
> >
> > Thanks to all the teams involved with developing, deploying, researching
> > and maintaining this service.
> >
> > P.S. This service was based off of prototypes Jon Robson had developed
> for
> > detecting trending articles. He will be continuing his work in this
> area. I
> > encourage you to reach out to him if you were interested in this project.
> >
> > [1] https://en.wikipedia.org/api/rest_v1/#!/Feed/trendingEdits
> > [2]
> > https://meta.wikimedia.org/wiki/Research:Comparing_most_
> read_and_trending_edits_for_Top_Articles_feature
> >
> >
> >
> > --
> > Corey Floyd
> > Engineering Manager
> > Readers
> > Wikimedia Foundation
> > cfloyd(a)wikimedia.org
> >
> --
> Corey Floyd
> Engineering Manager
> Readers
> Wikimedia Foundation
> cfloyd(a)wikimedia.org
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
Hello,
We're a small app development company that has integrated Wikipedia content
into a geo-locating iOS app. The app is working well and the Wiki content is
displaying correctly. However, we'd like to categorise the Wikipedia content
into three categories rather than just one.
Is there a way to filter and categorise Wikipedia content that is accessed
through the REST API? We only use content that is geo-coded (ie has latitude
and longitude) information associated with each article.
How should we go about configuring our API integration so that we can split
Wikipedia content according to its top-level categories? Is there a way to
do this?
Many thanks for your assistance with this request.
Regards,
Chris Smyth
Christopher Smyth
Director
Inflighto
chris(a)inflighto.com <mailto:chris@inflighto.com>
+61 (0)417 298 598
<https://www.inflighto.com/>