Hi,
people from gerrit's “Analytics” group [1] currently hold
* Push (including Force Push)
* Push Merge Commit
* Forge Author Identiy
* Forge Committer Identity
permissions on “analytics/*” projects in gerrit. But those permissions
got and get in the way one way or the other.
Do we need those permissions for our repos?
If no one objects, I'll start removing them on 2014-04-28.
Best regards,
Christian
[1] https://gerrit.wikimedia.org/r/#/admin/groups/uuid-d34747bee94be39cff54b5fd…
--
---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ----
Companies' registry: 360296y in Linz
Christian Aistleitner
Gruendbergstrasze 65a Email: christian(a)quelltextlich.at
4040 Linz, Austria Phone: +43 732 / 26 95 63
Fax: +43 732 / 26 95 63
Homepage: http://quelltextlich.at/
---------------------------------------------------------------
Hi folks,
As you've probably heard, last week we deployed ulsfo in production,
reducing latency for Oceania, East/Southeast Asia & US/Canada
pacific/west coast states. My estimation of the user base affected by
this is 360 million users (as in, Internet users, not Wikipedia users).
I was wondering if you have an easy way to measure and plot the impact
in page load time, perhaps using Navigation Timing data?
The operations team has spent a considerable amount of time and money to
deploy ulsfo and I believe it'd be useful for us and the organization at
large to be able to quantify this effort.
The exact dates of the rollout by country/region codes can be found in
operations/dns' git history:
https://git.wikimedia.org/summary/?r=operations/dns.git
(the commits should be self-explanatory, but I'd be happy to clarify if
needed)
Thanks!
Faidon
At about 2014-03-18 00:04 UTC, db1047 stopped accepting incoming
connections. At some point during the subsequent hour, MariaDB had either
crashed or been manually restarted. Sean noticed that the database was
choking on some queries from the researchers and notified the wmfresearch
list.
During the time that the database server was out or rejecting connection,
the EventLogging writer that writes to db1047 was repeatedly failing to
connect to it:
sqlalchemy.exc.OperationalError: (OperationalError) (2003, "Can't connect
to MySQL server on 'db1047.eqiad.wmnet' (111)")
The Upstart job for EventLogging is configured to re-spawn the writer, up
to a certain threshold of failures. Because the writer repeatedly failed to
connect, it hit the threshold, and was not re-spawned.
This triggered an Icinga alert:
[00:04:24] <icinga-wm> PROBLEM - Check status of defined EventLogging jobs
on vanadium is CRITICAL: CRITICAL: Stopped EventLogging jobs:
consumer/mysql-db1047
This alert was not responded to. I finally got pinged by Tillman, who
noticed the blog visitor stats report was blank, and by Gilles, who noticed
image loading performance data was missing.
We have to fix this. The level of maintenance that EventLogging gets is not
proportional to its usage across the organization. Analytics, I really need
you to step up your involvement.
It was not long ago that EventLogging was running reliably for months at a
time. What has changed is not system load, but the owner seat becoming
vacant, leading to a gradual deterioration of the quality of monitoring and
auditing practices.
Sean proposed moving the EventLogging database to m2, so that it runs on
separate hardware from the research databases. I think he's right. I filed <
https://rt.wikimedia.org/Ticket/Display.html?id=7081> to request the
migration.
There is some code rot around the Ganglia and Graphite monitoring code for
EventLogging. I don't think it would take much to fix. Could the Analytics
team take this on?
The Puppet code is well-documented. <
https://wikitech.wikimedia.org/wiki/EventLogging> could use some updating,
but it is mostly current.
Finally, I think EventLogging Icinga alerts should have a higher profile,
and possibly page someone. Issues can usually be debugged using the
eventloggingctl tool on Vanadium and by inspecting the log files on
vanadium:/var/log/upstart/eventlogging-*.
---
Ori Livneh
ori(a)wikimedia.org
Hi!
The speed bumps from the eventlogging migration are almost ironed out:
1. db1048 has had the eventlogging uuid fields made formally UNIQUE KEY. I
gather Ori will now run some validation against logs to check for remaining
gaps.
2. db1046 which died mid-migration has been restored and is catching up.
This doesn't really affect Analytics except that it's to be part of
db1047's replication chain for eventlogging.
3. db1047 is finishing up reloading log data and removing the CONNECT
federated tables involved in bug 64445[1].
As something of a consolation prize, "analytics-store.eqiad.wmnet" is now
open for SELECT queries from the 'research' user. This box:
- Is a CNAME for dbstore1002.eqaid.wmnet.
- Replicates all wikis in one place.
- Can be hammered. Please feel free.
- Can have scratch space for temporary writes (but doesn't yet).
- Can replicate eventlogging too (but doesn't yet).
I would appreciate if anyone has some suitable read-only reports to try
out, please do so and report back.
BR
Sean
[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=64445
--
DBA @ WMF
We are changing EventLogging to write events to m2 instead of db1047. The
migration will take up to 12 hours (but probably less). Also, we may end
up with gaps in the data written to the database throughout this period.
We will reply to this thread once the migration is complete.
Samuel Klein, 30/04/2014 05:35:
> Asking 1/1000 users of tool X a single open-ended question ("please
> give us feedback on X" or "how is X working for you"?) can be a handy
> way to encourage brief input from a cross-section of users,
We do have such a feature in MediaWiki, though: mediawiki.feedback.js.
It's just a JavaScrip popup which saves the comment to a page on the wiki.
> many of
> which would not otherwise comment at all. And for some tools (such as
> UploadWizard) there is no obvious place to leave comments, and opening
> Bugzilla is a new-tab + multi-step process away.
UploadWizard (like VisualEditor) uses what above. Maybe it needs an
option to be offered more prominently under some conditions?
This is probably the most viable option here, almost no technical effort
and more value in output.
Tilman Bayer, 29/04/2014 21:58:
> might be worth revisiting
> LimeSurvey, which appears to have undergone a complete rewrite since
> that installation was removed from WMF servers for security concerns
> around 2011.
+1. It will need to be done anyway, at some point, e.g. if a general
editor survey is tried again.
> multilingual
> support than other solutions [...]
> lack of integrated language support in
> Surveymonkey, or just because the focus was on per-project results
> anyway?
Agreed on all the rest but this point specifically. It seems
surveymonkey is really out of question. However, how many languages does
Qualtrics support?
LimeSurvey says 50; it is translated on a public instance of GlotPress.
GlotPress is from Automattic and is used to make some Wordpress locale,
hence some translatewiki.net have experience with it. However I wasn't
able to gather much information about it, I only know that it's yet
another web tool for .po format; maybe Stu can put us in contact with
someone with more insight (especially on how much it's used and how
prioritary for Automattic)?
Nemo
Some of the graphs [1] on the report card are not rendering due to
what seems like some sort of EventLogging outage
When I query the database I get errors such as ""MySQL said: Table
'MobileWebEditing_7675117' is marked as crashed and should be
repaired""
Let's keep an eye on this...
[1] http://mobile-reportcard.wmflabs.org/#edits_daily-graphs-tab
Hi,
Once upon a time, the Multimedia team set out on a great quest [0] to
add a survey to the Media Viewer they were building. This quest ended
with the team deciding that the great Lord SurveyMonkey would provide
the users the survey they needed with the least amount of effort, and
there was much rejoicing.
But a dark threat loomed over the land. With one product using SurveyMonkey,
other products seemed poised to use it, too [1]. The compromise built
upon the premise that Media Viewer needed a survey in less time than it
would take to find and set up a free [2] solution was beginning to bleed
over into other projects where no such time crunch was present.
Our heroes now reach out to their friends in other realms [3]. Is there
hope for freedom in the land of getting user feedback? Will MediaWiki
or the grander Wikimedia ecosystem soon have a survey tool that all projects
can use with minimal hassle?
I'd appreciate looking at existing solutions and planned non-existing
ones, and I offer some of my ample [4] free time to help with setting
something up.
[0] https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/261
[1] https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/522
[2] https://gnu.org/philosophy/free-sw.html
[3] http://mediawiki.org/wiki/Analytics
[4] https://en.wikipedia.org/wiki/Sarcasm
Cheers,
--
Mark Holmquist
Software Engineer, Multimedia
Wikimedia Foundation
mtraceur(a)member.fsf.org
https://wikimediafoundation.org/wiki/User:MHolmquist
(cross-posting, slightly edited to add more context)
I uploaded several plots with absolute editor activation counts and editor activation rates. These are based on data we generated for the sensitivity analysis of “new user” metrics. [1]
https://meta.wikimedia.org/wiki/Research:Editor_activation#Plots
These charts are roughly what you should expect from the EEVS project [2]. They assume the ability to (1) compute rates on top of absolute counts, (2) aggregate monthly, not just daily, and (3) combine multiple series in the same chart — i.e., not necessarily all features included in the minimum viable product.
As usual, feedback is very welcome (particularly on the way of presenting this data that you find most useful).
Dario
[1] https://meta.wikimedia.org/wiki/Research:Metrics_standardization#New_users
[2] https://www.mediawiki.org/wiki/Analytics/Epics/Editor_Engagement_Vital_Signs