This is a little inferior to the production version as
it is unable to use
production kafka and if it has any outages it will lose data.
EventStreams isn’t as good as using Kafka, but an outage shouldn’t be a
reason to lose data. Store the Last-Event-ID
<https://wikitech.wikimedia.org/wiki/EventStreams#Format> that EventStreams
gives you, and you can use it when the service starts back up to start from
where you left off.
> and maybe, as others have mentioned, a good reason to get production
Kafka events flowing into Cloud VPS backed projects.
Def not opposed to a Kafka cluster in Cloud mirroring from Prod. :)
BTW, this is a wee relevant:
https://wikitech.wikimedia.org/wiki/User:Ottomata/Stream_Data_Platform
This is a draft! I’m shopping this around as a program for next FY. We
will see!
On Tue, Dec 12, 2017 at 4:16 PM, Gergo Tisza <gtisza(a)wikimedia.org> wrote:
On Tue, Dec 12, 2017 at 12:12 PM, Jon Robson
<jdlrobson(a)gmail.com> wrote:
This is a little inferior to the production
version as it is unable to
use
production kafka and if it has any outages it
will lose data.
Hopefully that gets fixed soon, Cloud VPS / Toolforge is the foundation for
out volunteer tool developer community who really shouldn't be treated as
second-class citizens.
Other than that, moving to the Cloud is not a bad thing for an experimental
project IMO. It makes it easier to experiment with minimal risk, and it
makes it easy to add co-collaborators to your project without having to get
prod access for them.
I'm hoping to get this onto IFTTT <https://ifttt.com/wikipedia> with help
from Stephen Laporte in my volunteer time, as I
think this feature is a
pretty powerful one which has failed to find its use case in the wiki
world. As Kaldari points out it's incredibly good at detecting edit wars
and I personally have learned a lot about what our editors see as
important
and notable in the world (our editors really seem
to like wrestling). I
think there are ample and exciting things people could build on top of
this
api.
Yeah a "give me push notifications about ongoing edit wars" tool for admins
sounds really cool. Although you'd probably want to look at revert trends
(or both edit and revert trends) for that.
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l