It would be nice if MediaWiki API _AND_ pywikipedia bot do not deprecate
at once.
Now it looks as
API: we are deprecating what we promised to deprecated long ago - ok
pywikipedia compat: did not handle the deprecation of API before, and are
not going to fix copy-pasted in tens of places (not one place, it's never
that simple) query builders to support "rawcontinue", we announce compat as
discontinued together with the old style API.
API deprecation was not coordinated with client library deprecation - not ok
If there is one year gap between two deprecations - ok, bot writers can
choose either compat or core, and their bots can still work.
Most users don't use APi directly so it should be the problem of
coordination between API and clients developers.
----------------------------------------------------------------------
Message: 1
Date: Wed, 3 Jun 2015 19:08:24 +0300
From: Yuri Astrakhan <yastrakhan(a)wikimedia.org>
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation
mode for action=query will change at the end of this month
Message-ID:
<CALOOOkhzCUf14Tf=
P0uQw7ZXAmwb9qAxs4kgvbJAvSsMQY82GA(a)mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
MZ: >I feel that former MediaWiki Web API maintainers should actively pay
attention to which mailing lists they're posting to. ;-) I doubt you
intended to send this message to mediawiki-api-announce.
Mz, I don't think I ever spent much time maintaining it :)) But yes, good
point, reply all is evil at times :)
MZ: re why minimalistic lib - for most apis out there, people tend to write
"reference implementation" - code that explains to would-be lib authors how
to use API. It sets up prefered usage patterns and guidelines. We never had
that for the api. This resulted in a large number of API libs that vary in
how they use api. Pywikibot is a powerful platform, but is too big for many
usecases (as discussed in Lyon). Hence the need.
SW: >Most operator are volunteers and don't have time to change the code
every month because there is a change in the api.
* I might agree about every month, but we are talking about a feature that
has been out for 2 years...
The old one was imho perfectly.
* Most API
users imho would laugh at this statement. See the roadmap
<https://www.mediawiki.org/wiki/Requests_for_comment/API_roadmap> which
came as a result of analysing numerous bugs & wishes filed against the api,
such as returning {} instead of [] for empty values, inconsistencies, the
silly '*' value for text, and many other problems that have accumulated
during the years. API might work for you, but it does not support
multilingual error messaging, it overcomplicates the process of writing
javascript clients, it does not easily cache. In short, lots of problems.
Was the new api coded by WMF or by volunteers?
* I wrote the original API as a volunteer (took about a year in 2005/6). I
recently coded the continuation as a volunteer, and Brad has improved it as
a WMF employee. I later became a full time WMF employee as well, but my
project was Zero, not API. As a volunteer, over 2 years ago I wrote a huge
API
improvement roadmap
<https://www.mediawiki.org/wiki/Requests_for_comment/API_roadmap>that Brad
picked up and greatly extended, and is driving it forward with the amazing
improvements. From what I know, Brad has now been officially moved into
position of working on the API. In other words, that employee vs volunteer
line is very fuzzy. No point of splitting it that way.
TLDR;
In short, we need to make improvements if we want to move forward in
turning API into a platform, with flexible JavaScript-driven interfaces
such as Visual Editor. To allow creative uses that go beyond AWB, that
support complex interactions, and not just the way for bots to make edits.
Unfortunately, that means that despite our best efforts, very infrequently,
some bots must be updated.
If the bot author cannot add a simple one line change "rawcontinue=1" once
in two years because of their busy personal live, I don't think that person
should be making automatic edits to Wikipedia - because sometimes bots make
mistakes that require substantially more time involvement. I would not
trust wikipedia edits to a bot runner under such circumstances. If the bot
runner is not a programmer, they should get the latest version of their
bot. If there is no up-to-date code because noone is maintaining it, it
again should not be accessing wikipedia - we sometimes discover security
bugs that require fixing, or because bot calls wiki too often, or other
changes in content structure - e.g. introduction of WikiData for interwiki
links required all interwiki bots to be updated.
Wikipedia is a living, developing ecosystem, and it does require updates to
all parties accessing it. People accessing wikipedia from the older
browsers discover that they no longer can do everything they used to many
years ago - because we now use newer browser features, and fallback into
basic mode for older browsers. Please participate in the evolution of the
project.
On Wed, Jun 3, 2015 at 5:22 PM, Steinsplitter Wiki <
steinsplitter-wiki(a)live.com> wrote:
Most operator are volunteers and don't have
time to change the code every
month because there is a change in the api. Because of this devs should
keep the api backward-compatible.
Also wondering why wee need this "new" api. The old one was imho
perfectly.
>
Was the new api coded by WMF or by volunteers?
>
> > I feel that bot operators should actively pay attention to the
technical
aspects
of the community and the mailing lists.
Sorry, i disagree. Bot operators are
volunteers and not payed staffers.
Most of them having a job and real live.
-- Steinsplitter
> Date: Wed, 3 Jun 2015 14:50:48 +0300
> From: yastrakhan(a)wikimedia.org
> To: wikitech-l(a)lists.wikimedia.org
> CC: mediawiki-api-announce(a)lists.wikimedia.org
> Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation
mode
for action=query will change at the end of this
month
>
> I feel that bot operators should actively pay attention to the
technical
> aspects of the community and the mailing
lists. So, the bot operator
who
> never updates their software, doesn't
pay attention to the
announcements,
> and ignores api warnings should be blocked
after the deadline. Bot
> operators do not operate in a vacuum, and should never run bots just
for
the sake
of running them.
Community should always be able to find and communicate with the bot
operators.
Obviously we should not make sudden changes (except in the
security/breaking matters), and try to make the process as easy as
possible. The rawcontinue param is exactly that, simply adding it will
keep
> the logic as before.
>
> Lastly, I again would like to promote the idea discussed at the
hackathon
-- a
client side minimalistic library that bigger frameworks like
pywikibot
rely on, and that is designed in part by the core
developers. See the
proposal at
https://www.mediawiki.org/wiki/Requests_for_comment/Minimalistic_MW_API_Cli…
> On Jun 3, 2015 2:29 PM, "John Mark
Vandenberg" <jayvdb(a)gmail.com>
wrote:
> On Wed, Jun 3, 2015 at 3:42 AM, Brad Jorsch (Anomie)
> <bjorsch(a)wikimedia.org> wrote:
> > ...
> > I've compiled a list of bots that have hit the deprecation warning
more
> > than 10000 times over the course of the
week May 23–29. If you are
> > responsible for any of these bots, please fix them. If you know who
is,
> > > please make sure they've seen this notification. Thanks.
> >
> > Thank you Brad for doing impact analysis and providing a list of the
> > 71 bots with more than 10,000 problems per week. We can try to solve
> > those by working with the bot operators.
> >
> > If possible, could you compile a list of bots affected at a lower
> > threshold - maybe 1,000. That will give us a better idea of the
scale
> > of bots operators that will be affected
when this lands - currently
in
one months time.
Will the deploy date be moved back if the impact doesnt diminish by
bots being fixed?
--
John Vandenberg
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
------------------------------
Message: 2
Date: Wed, 3 Jun 2015 12:34:12 -0400
From: "Brad Jorsch (Anomie)" <bjorsch(a)wikimedia.org>
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation
mode for action=query will change at the end of this month
Message-ID:
<
CAEepRSuKej9O0beGt6xco+c+j7uTa0BtZoeCAQs9Cz6XQ2gYwg(a)mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
On Wed, Jun 3, 2015 at 6:49 AM, Steinsplitter Wiki <
steinsplitter-wiki(a)live.com> wrote:
I haven't followed this discussion, however i
am wondering why api is not
keep backward compatible. This will break a lot of stuff and i am
wondering
why we need such changes in the [API]
We usually do. In this case, however, the advantages of changing the
default for new API users seems to outweigh the disadvantages of a
well-announced breaking change with a simple parameter to request
backwards-compatible output.
--
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
------------------------------
Message: 3
Date: Wed, 3 Jun 2015 12:43:36 -0400
From: "Brad Jorsch (Anomie)" <bjorsch(a)wikimedia.org>
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
Cc: mediawiki-api-announce(a)lists.wikimedia.org
Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation
mode for action=query will change at the end of this month
Message-ID:
<
CAEepRSvprWamR3y+TFfcLg6Jv4hkbtCqbzj6JxfSvZe4v8f4Ow(a)mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
On Wed, Jun 3, 2015 at 7:29 AM, John Mark Vandenberg <jayvdb(a)gmail.com>
wrote:
If possible, could you compile a list of bots
affected at a lower
threshold - maybe 1,000. That will give us a better idea of the scale
of bots operators that will be affected when this lands - currently in
one months time.
I already have the list of *accounts* affected: there are 510 with between
1000 and 10000 hits. Of those, 454 do not contain "bot" (case
insensitively), so they might be human users with user scripts, or AWB if
that's not fixed (someone please check!), or the like. For comparison, in
the over-10000 group there were 30 such that I filtered out.
I'll want to check with Legal to make sure the additional release of
account names is still compliant with the privacy policy (I'm almost but
not entirely sure it would be ok).
Will the deploy date be moved back if the impact
doesnt diminish by
bots being fixed?
That's not impossible, but I wouldn't count on it.
--
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
------------------------------
Message: 4
Date: Wed, 3 Jun 2015 13:13:34 -0400
From: "Brad Jorsch (Anomie)" <bjorsch(a)wikimedia.org>
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
Cc: MediaWiki API announcements & discussion
<mediawiki-api(a)lists.wikimedia.org>
Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation
mode for action=query will change at the end of this month
Message-ID:
<CAEepRSsfin=praEeth2qQQFtwS_SHgFDbZFpXEd=
uqORvn4sJA(a)mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
On Wed, Jun 3, 2015 at 10:04 AM, Brian Gerstle <bgerstle(a)wikimedia.org>
wrote:
My question is: why does the default behavior
need to change? Wouldn't
continuing with the default behavior allow people to continue using the
"rawcontinue" behavior for as long as we want to support it—without
making
any changes?
The decision to change the default came out of the same concerns that led
to the improved action=help output and some of the other work I've been
doing lately: We want to lower the barriers for using our API, which means
that the default shouldn't be something user-hostile.
The raw continuation is deceptively simple: it looks straightforward, but
if you're using it with a generator, multiple prop modules, and meta or
list modules, your client code has to know when to ignore the returned
continuation for the generator, when to remove a module from prop and then
when to re-add it, and when to remove the meta or list modules. I wouldn't
be that surprised to learn that more people have it wrong than correct if
their code supports using prop modules with generators.
The new continuation actually is simple: you send the equivalent of
array_merge( $originalParams, $continueParams ) and it just works.
Yes, some of the same could be said for making format=json&formatversion=2
the default. In this case the formatversion=1 output is just annoying
rather than actually hostile (although representing boolean true as a
falsey string comes close), so at this time there's no plan to make that
breaking change.
--
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
------------------------------
Message: 5
Date: Wed, 3 Jun 2015 20:31:42 +0300
From: Yuri Astrakhan <yastrakhan(a)wikimedia.org>
To: "MediaWiki API announcements & discussion"
<mediawiki-api(a)lists.wikimedia.org>
Cc: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
Subject: Re: [Wikitech-l] [Mediawiki-api] API BREAKING CHANGE: Default
continuation mode for action=query will change at the end of this
month
Message-ID:
<
CALOOOkjcnXoXCSMOQe5z58NR2iw24tf7qPcOkSXWT1qZLt30hQ(a)mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
I would like to point out that it might be a good idea add
&formatversion=1 for anyone who wants to lock the current formatting in
place.
On Wed, Jun 3, 2015 at 8:13 PM, Brad Jorsch (Anomie) <
bjorsch(a)wikimedia.org>
wrote:
On Wed, Jun 3, 2015 at 10:04 AM, Brian Gerstle
<bgerstle(a)wikimedia.org>
wrote:
> My question is: why does the default behavior need to change? Wouldn't
> continuing with the default behavior allow people to continue using the
> "rawcontinue" behavior for as long as we want to support it—without
making
any
changes?
The decision to change the default came out of the same concerns that led
to the improved action=help output and some of the other work I've been
doing lately: We want to lower the barriers for using our API, which
means
that the default shouldn't be something
user-hostile.
The raw continuation is deceptively simple: it looks straightforward, but
if you're using it with a generator, multiple prop modules, and meta or
list modules, your client code has to know when to ignore the returned
continuation for the generator, when to remove a module from prop and
then
when to re-add it, and when to remove the meta or
list modules. I
wouldn't
be that surprised to learn that more people have
it wrong than correct if
their code supports using prop modules with generators.
The new continuation actually is simple: you send the equivalent of
array_merge( $originalParams, $continueParams ) and it just works.
Yes, some of the same could be said for making
format=json&formatversion=2
the default. In this case the formatversion=1
output is just annoying
rather than actually hostile (although representing boolean true as a
falsey string comes close), so at this time there's no plan to make that
breaking change.
--
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
_______________________________________________
Mediawiki-api mailing list
Mediawiki-api(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-api
------------------------------
Message: 6
Date: Wed, 3 Jun 2015 11:55:40 -0600
From: Brian Wolff <bawolff(a)gmail.com>
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>rg>,
mediawiki-api(a)lists.wikimedia.org
Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation
mode for action=query will change at the end of this month
Message-ID:
<
CA+oo+DUnkpwL_EsG7DmGQOq5OsSPJmH9FnL+eW_xbaOO6XFjdg(a)mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
On 6/3/15, Steinsplitter Wiki <steinsplitter-wiki(a)live.com> wrote:
Most operator are volunteers and don't have
time to change the code every
month because there is a change in the api. Because of this devs should
keep
the api backward-compatible.
Also wondering why wee need this "new" api. The old one was imho
perfectly.
>
Was the new api coded by WMF or by volunteers?
>
>> I feel that bot operators should actively pay attention to the technical
>> aspects of the community and the mailing lists.
> Sorry, i disagree. Bot operators are volunteers and not payed staffers.
Most
of them having a job and real live.
-- Steinsplitter
My understanding is that most of the people who were using the
original continuation, were using it wrong, causing subtle bugs in
their script. Thus the existing implementation was wasting
considerable amount of volunteer bot developer time. In the long run
this change will hopefully reduce the total amount of time spent by
volunteer bot makers chasing weird bugs, at the expense of some short
term pain.
Its always a challenge to balance backwards compatibility with fixing
things that are causing problems. I think the API team is keenly aware
of the frustrations that changes to the api cause, and try to make
sure that intentional breakage only happens when the benefits truly
outweigh the cons.
--
Bawolff
------------------------------
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
End of Wikitech-l Digest, Vol 143, Issue 8
******************************************