Hi all,
We finished our sprint on Tuesday and made plans for the next one on
Thursday and I wanted to let you know the updates.
We've been in the middle of annual planning, so I'm going to provide the
update on Epics next week.
We finished the two projects that we had committed to for the sprint:
Fundraising Visualization
Syncup<https://wikimedia.mingle.thoughtworks.com/projects/analytics/cards/1449>
Measuring ULSFO Site
Performance<https://wikimedia.mingle.thoughtworks.com/projects/analytics/cards/1450>
We were also able to finish:
Productionize Wikimetrics<https://wikimedia.mingle.thoughtworks.com/projects/analytics/cards/1185>
We generated the dashboards for the March metrics meeting
We wrote a utility (kraken-toolbelt) that allows read/write from/to snappy
compressed hdfs files
We fixed the following defects:
Adapt to tag change in comScore csv
files<https://bugzilla.wikimedia.org/show_bug.cgi?id=62044>
Merge editors stats for all wikis failed
No sampled-1000 tsv file for 2014-02-06 on
stat1002<https://bugzilla.wikimedia.org/show_bug.cgi?id=60955>
We also worked on, but did not finish a few other issues:
Run Scheduled Reports via
Wikimetrics<https://wikimedia.mingle.thoughtworks.com/projects/analytics/cards/1376>
Unit testing Hive queries
Kafka-Tee testing/deployment
For the new sprint, we needed to pivot and migrate our existing services to
the new labs cluster in the Virginia datacenter. We've tasked this out, but
there are too many unknowns and possible dependencies to properly estimate,
so we will work on this until it's done.
-Toby
Hello,
I watch pageview traffic to about 100 popular Wikipedia health articles. In
January 2014 as compared to January 2013, stats.grok.se reports about a 50%
drop in pageviews.
Does someone have a ready explanation for this? The obvious explanation
would be that Wikipedia's traffic has halved, which does not seem right to
me, but it could be so.
Can someone direct me to previous or current conversations about this?
yours,
--
Lane Rasberry
user:bluerasberry on Wikipedia
206.801.0814
lane(a)bluerasberry.com
Hi,
with the Analytics Development team's decision to move from dclass to
ua-parser, life support for
http://stat1001.wikimedia.org/public-datasets/analytics/mobile/mobile_devic…http://stat1001.wikimedia.org/public-datasets/analytics/mobile/mobile_platf…
would become a burden. As we additionally do not know of any user of
those files up to now, we are considering to stop generating those two
files [1].
If you are using any of the above two files, please let us know by
2014-03-05, so we can discuss how to move forward.
Best regards,
Christian
[1] It's really only those two files listed above.
We keep on generating all the other mobile files as for example the
mobile-sampled-100 as usual for the time being.
--
---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ----
Companies' registry: 360296y in Linz
Christian Aistleitner
Gruendbergstrasze 65a Email: christian(a)quelltextlich.at
4040 Linz, Austria Phone: +43 732 / 26 95 63
Fax: +43 732 / 26 95 63
Homepage: http://quelltextlich.at/
---------------------------------------------------------------
OpenPGP key transition from 0xEF78CCDE to 0x13C1072F:
http://quelltextlich.at/openpgp-transition-0xEF78CCDE-to-0x13C1072F.txt
Hi,
this table:
http://stats.wikimedia.org/wikispecial/EN/TablesWikipediaWIKIDATA.htm#editd…
seems to think that there have only been 15,238,669 edits by non-IPs in the
article namespace on Wikidata. Considering Wikidata crossed 100M edits
overall, I would be extremely suprised if either IPs or non-article edits
would make up the difference.
Can someone check what is going on there? Especially with regards to how
trustworthy the rest of the linked table is?
Cheers,
Denny
Yoyo,
Who generates the mobile and zero limn dashboards? I am currently generating log files from Kafka that should be useable in place of the ones that udp2log generates. It’d be cool to compare the output of the two.
Christian has already done a little bit of comparison, now it’d be nice to see the output used for something! :)
Hi,
it seems that actions on our end are necessary within a month for our
labs instances to survive the data center migration (see below).
I am all for migrating our instances (at least limn0, as it is
required for Wikipedia Zero).
But since I cannot find a Mingle card for the labs migration, and it
was not brought up before:
Do we want to migrate our instances?
Best regards,
Christian
----- Forwarded message from Andrew Bogott <abogott(a)wikimedia.org> -----
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>, "A list for announcements and discussion related to the Wikimedia Labs project."
<labs-l(a)lists.wikimedia.org>
From: Andrew Bogott <abogott(a)wikimedia.org>
Date: Fri, 28 Feb 2014 23:59:00 +0800
Subject: [Wikitech-l] Labs migration starts on Tuesday
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.3.0
List-Id: Wikimedia developers <wikitech-l.lists.wikimedia.org>
Starting on Tuesday, March 4th, the new Labs install in the eqiad data
center will be open for business. Two dramatic things will happen on
that day: Wikitech will gain the ability to create instances in
eqiad, and wikitech will lose the ability to create new instances in
pmtpa.
About a month from Tuesday, the pmtpa labs install will be shut down.
If you want your project to still be up and running in April, you must
take action!
We are committed to not destroying any instances or data during the
shutdown, but projects that remain untouched by human hands during the
next few weeks will be mothballed by staff: the data will be preserved
but most likely compressed and archived, and instances will be left in
a shutdown state.
(Note: Toollabs users can sit tight for a bit; Coren will provide
specific migration instructions for you shortly.)
I've written a migration guide, here:
https://wikitech.wikimedia.org/wiki/Labs_Eqiad_Migration_Howto It's a
work in progress, so check back frequently. Please don't hesitate to
ask questions on IRC, make suggestions as to guide improvements, or
otherwise question this process. Quite a few of the suggested steps
in that guide require action on the part of a Labs op -- for that
purpose we've created a bugzilla tracking bug, 62042. To add a
migration bug that links to the tracker, use this link: https://bugzilla.wikimedia.org/enter_bug.cgi?product=Wikimedia%20Labs&compo…
At the very least, please visit this page and edit it with your
project migration plans:
https://wikitech.wikimedia.org/wiki/Labs_Eqiad_Migration_Progress
Projects that have no activity on that page will be early candidates
for mothballing. If you want me to delete your project, please note
that as well -- that will allow us to free up resources for future
projects.
I am cautiously optimistic about this migration. Most of our testing
has gone fairly well, so a lot of you should find the process smooth
and easy. That said, we're all going to be early adopters of this
tech, so I appreciate your patience and understanding when inevitable
bugs shake out. I look forward to hearing about them on IRC!
-Andrew
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
----- End forwarded message -----
--
---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ----
Companies' registry: 360296y in Linz
Christian Aistleitner
Gruendbergstrasze 65a Email: christian(a)quelltextlich.at
4040 Linz, Austria Phone: +43 732 / 26 95 63
Fax: +43 732 / 26 95 63
Homepage: http://quelltextlich.at/
---------------------------------------------------------------
Dear fellow Analytics Developer team members,
over the past few weeks, it seems we at least twice discussed that
maybe we want to adopt “If it didn't happen on the list, it didn't
happen”. It appeared to me that both times, we actually wanted to move
forward with that ... but it never got posted to the mailing list :-D
So I'm being bold:
Either slap me with a large trout by 2014-02-28 or agree with me
that the Analytics development team adopts “If it didn't happen on
the list, it didn't happen.”
(With “list” being analytics(a)lists.wikimedia.org for public things,
and “list” being analytics-internal(a)lists.wikimedia.org for not so
public things.)
To state the obvious:
meetings in real life,
hangouts, or
personal email
do not qualify as “list” :-)
Have fun,
Christian
/me hands a large trout to all of you to slap me with and ducks.
--
---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ----
Companies' registry: 360296y in Linz
Christian Aistleitner
Gruendbergstrasze 65a Email: christian(a)quelltextlich.at
4040 Linz, Austria Phone: +43 732 / 26 95 63
Fax: +43 732 / 26 95 63
Homepage: http://quelltextlich.at/
---------------------------------------------------------------
OpenPGP key transition from 0xEF78CCDE to 0x13C1072F:
http://quelltextlich.at/openpgp-transition-0xEF78CCDE-to-0x13C1072F.txt