I've collected some info of Gather API calls the Android app could use to
sync saved pages. This could be extended for general collections and watch
lists later.
There are some features missing from the Gather API and deployments to
other wikis besides en and he WP. See my comments on
https://phabricator.wikimedia.org/T123377 for details. Add your own
comments if you have ideas for improvement or can help fix any of the
current limitations of the API.
Thanks,
Bernd
>Pywik up and running for iOS (simple machine spin-up, if not finished in Q2
FYI that we have a preliminary install of piwik in production that we can
use for IOS Application[1]
Please let us know when you are ready to switch IOS app to this instance
and stop using labs.
Thanks,
Nuria
[1] https://wikitech.wikimedia.org/wiki/Analytics/piwik
---------- Forwarded message ----------
From: Jon Katz <jkatz(a)wikimedia.org>
Date: Fri, Dec 4, 2015 at 12:25 PM
Subject: Re: [Analytics] Preliminary goals for analytics infrastructure team
To: "A mailing list for the Analytics Team at WMF and everybody who has an
interest in Wikipedia and analytics." <analytics(a)lists.wikimedia.org>
Hi Nuria,
Thank you for looking at these. I think your assessment is a fair
representation of the asks. Just adding a new one (in green).
- Pywik up and running for iOS (simple machine spin-up, if not finished
in Q2)
- Input pageview data into tableau or some other non-query table maker
- 'is_spider' column in event logs (Based on regex of user agent- based
on conversation with Madhu and Kevin, I know its not going to get most of
the bots)
- App session tables split by OS (per ticket: T117615
<https://phabricator.wikimedia.org/T117615>)
- Time on site (per: T119352 <https://phabricator.wikimedia.org/T119352>
)
- Browser reports you mention in your current priorities (this is low
for us if it is in hive already, though you probably have other
stakeholders)
- connection speed field...somewhere ;)
Also, to be clear, I am assuming that: Unique users/sessions are considered
to be outside the scope of this prioritization. Let me know if I'm
mistaken. I believe Toby will be scheduling some time to talk through the
above at some point as well.
-J
On Thu, Dec 3, 2015 at 8:55 AM, Nuria Ruiz <nuria(a)wikimedia.org> wrote:
> Some thoughts below (we will be creating tickets for some of these so we
> do not forget)
>
> > connection speed field...somewhere ;)
> I think many teams will benefit from this one but as far as I can see
> there is no way to get this info (and it being correct) at this time.
> Performance team just recently opened a ticket in this regard and we saw it
> was not possible. See: https://phabricator.wikimedia.org/T119801 and
> http://www.w3.org/TR/netinfo-api/
>
>
> >'is_spider' column in event logs (Based on regex of user agent- based on
> conversation with Madhu and Kevin, I know its not going to get most of the
> bots
> For eventlogging data? On the contrary, it will get most of the bot
> traffic. This is not true of our pageview data and I can explain in more
> detail via IRC why is that. Please let me know if I misunderstood
> this referring to EL.
>
> >App session tables split by OS (per ticket: T117615
> <https://phabricator.wikimedia.org/T117615>)
> Agreed. This should be part of the background work we do that is not
> related to goals.
>
> .>Piwik up and running for iOS (simple machine spin-up, if not finished
> in Q2)
> This would be very useful for other things besides IOS but it is harder
> than it looks as holding production data requires proper infrastructure (to
> be able to abide to our privacy policy) and to be honest I do not think we
> can get there next quarter. I think Dan can add more detail here as we have
> tried to use Piwik before outside labs w/o success.
>
>
> >Input pageview data into tableau or some other non-query table maker
> Need to think more about this use case.
>
>
>
>
>
>
> On Wed, Dec 2, 2015 at 9:16 PM, Jon Katz <jkatz(a)wikimedia.org> wrote:
>
>> Hi Nuria,
>> Thank you for publishing your draft so early and requesting feedback. I
>> apologize for being so late in responding--the short holiday week caused a
>> serious backlog on my end.
>>
>> For now, I only have a preliminary prioritized list of reading requests
>> for analytics in Q3 (and beyond). I can confirm by Friday. The following
>> are in order of priority
>>
>> - Pywik up and running for iOS (simple machine spin-up, if not
>> finished in Q2)
>> - Input pageview data into tableau or some other non-query table maker
>> - 'is_spider' column in event logs (Based on regex of user agent-
>> based on conversation with Madhu and Kevin, I know its not going to get
>> most of the bots)
>> - App session tables split by OS (per ticket: T117615
>> <https://phabricator.wikimedia.org/T117615>)
>> - Browser reports you mention in your current priorities (this is low
>> for us if it is in hive already, though you probably have other
>> stakeholders)
>> - connection speed field...somewhere ;)
>>
>> I'd be happy to discuss more either via chat or email.
>> Best,
>>
>> J
>>
>>
>> On Tue, Nov 24, 2015 at 8:49 AM, Nuria Ruiz <nuria(a)wikimedia.org> wrote:
>>
>>> Hello,
>>>
>>> Please see preliminary goals for the analytics infrastructure team next
>>> quarter. Replies to this thread with suggestions and feedback are welcome.
>>>
>>>
>>> https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q3_Goals#Analy…
>>>
>>>
>>> Thanks,
>>>
>>> Nuria
>>>
>>> _______________________________________________
>>> Analytics mailing list
>>> Analytics(a)lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/analytics
>>>
>>>
>>
>> _______________________________________________
>> Analytics mailing list
>> Analytics(a)lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/analytics
>>
>>
>
> _______________________________________________
> Analytics mailing list
> Analytics(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
>
>
_______________________________________________
Analytics mailing list
Analytics(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics
FYI. Tilman is in discussion with Nuria for remediation of this in order to
get things back up and running to support analysis on this event set.
---------- Forwarded message ----------
From: Nuria Ruiz <nuria(a)wikimedia.org>
Date: Wed, Jan 13, 2016 at 2:49 PM
Subject: [Analytics] MobileWebSectionUsage table dropped from Eventlogging
mysql db. Data is available on hadoop.
To: "A mailing list for the Analytics Team at WMF and everybody who has an
interest in Wikipedia and analytics." <analytics(a)lists.wikimedia.org>
Cc: Jon Robson <jdlrobson(a)gmail.com>
Team:
We had to take some desperate measures to unblock replication on
Eventlogging in the absence of our DBA.
We had to drop the MobileWebSectionUsage table. Table is blacklisted as
stream of events was to great for the system to sustain it, we will
whitelist it again once the new sampling rate takes effect.
Data is not available in mysql but it is present on hadoop so , to be
clear, no data has been lost.
We will report on replication issues as we have more knowledge. Sorry again
about the late notice.
Thanks,
Nuria
_______________________________________________
Analytics mailing list
Analytics(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics
In case you don't follow wikitech-l.
-Adam
---------- Forwarded message ----------
From: Brad Jorsch (Anomie) <bjorsch(a)wikimedia.org>
Date: Thu, Jan 14, 2016 at 8:55 AM
Subject: [Wikitech-l] MediaWiki authentication changes
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
*TL;DR:* Stuff is changing. If you notice problems with session handling in
1.27.0-wmf.11, file a bug in Phabricator and CC me (Anomie) and Gergő (Tgr).
MediaWiki's authentication layer is getting a major overhaul! This will
allow us to do cool stuff like 2-factor authentication without the 2-factor
extension having to be redone for LdapAuth/CentralAuth/every-other-auth,
third-party login (e.g. Google, Facebook) without hacking around everything
with crazy hooks, and supporting more than just one authentication method
at a time.
Broadly speaking, this change involves the addition of two new things:
SessionManager (already merged, being deployed with 1.27.0-wmf.11) and
AuthManager (hopefully coming by the end of February).
- SessionManager replaces the use of PHP's $_SESSION and session_*()
functions and the UserLoadFromSession and UserSetCookies hooks with a
more
extensible mechanism.
- AuthManager allows for independent pluggable authentication modules,
multiple authentication methods, and removes the assumption that all
logins
can be handled with a one-page form having "username" and "password"
fields. No more needing to adjust every authentication extension to know
about every other authentication extension it might have to cooperate
with.
However, the changes we're making do mean that various things are going to
need updating.
== For users ==
You shouldn't notice any changes due to SessionManager. If you do see
problems related to session handling in 1.27.0-wmf.11, file a bug in
Phabricator and CC me (Anomie) and Gergő (Tgr).
Once AuthManager comes around, the most visible change will be that the
login form will probably be a bit different, and if your browser remembers
your login information for you then you might have to re-enter it.
== For bots ==
You shouldn't notice any changes due to SessionManager, but do make sure
your code is handling cookies normally instead of using the deprecated
return values from action=login to construct cookies manually.[1] If you
are handling cookies properly and see problems related to session handling
in 1.27.0-wmf.11, file a bug in Phabricator and CC me (Anomie) and Gergő
(Tgr).
For AuthManager, the new features mean that unattended login might no
longer work since the login flow will now natively support user
interaction: the account might have 2-factor enabled, or might need a
password reset, or some other thing that requires user interaction. We've
created two ways to work around this:
- If possible, switch to OAuth. This week (1.27.0-wmf.10) "owner-only"
consumers are being rolled out to make this easier for bot operators: log
into your bot account, go to
https://meta.wikimedia.org/wiki/Special:OAuthConsumerRegistration/propose
,
and create a consumer with the "This consumer is for use only by
MyBotName"
checkbox checked.The consumer will be approved for use immediately, no
waiting or trying to find someone who can approve the consumer for you.
Owner-only consumers also don't tag every edit, since all the edits will
be
from the one account anyway.
- If you need to continue using the existing action=login, next week
(1.27.0-wmf.11) we're rolling out Bot Passwords. This is something like
OAuth-lite, or Google's application passwords: go to
Special:BotPasswords,
set one up, and then use new bot-password username and password to login
as
you've always done (no code changes, just update your bot's
configuration).
It's already live in Beta Labs if you want to test it out.
Login with the "main" account password will continue to work until
AuthManager is deployed, and might continue to work after that too (as long
as nothing requires user interaction).
For bots that run on third-party wikis, Bot Passwords are in core and are
enabled by default, but it's possible a wiki could disable them. OAuth is
an extension.
[1]:
https://lists.wikimedia.org/pipermail/mediawiki-api-announce/2015-December/…
== For user scripts and gadgets ==
Since user scripts and gadgets rely on the user already being logged in,
you shouldn't notice any changes.
== For core and extension developers ==
There's some other useful stuff that has already been merged, written as
part of the groundwork for this project:
- User::newSystemUser(), which replaces directly screwing around with
accounts when an extension needs a user for logged actions.
- Deprecation or removal of much of the password handling logic from the
User class. In particular User::getPassword() no longer works.
- CentralIdLookup, a generic mechanism for looking up a "central" ID for
cross-wiki features instead of everything needing that functionality
having
to depend on CentralAuth (and on anything else that might implement
global
accounts). If you have some sort of cross-wiki login extension that isn't
CentralAuth, you'll want to implement this class.
If you're dealing with session data,
- Don't use $_SESSION. Instead, use $request->getSession() or
SessionManager::getGlobalSession() to fetch a Session object and access
the
data through it. Or continue to use the $request->getSessionData() and
$request->setSessionData() methods.
- Don't use session_id() to test if a session is active. Instead, fetch
a Session object as above and call ->isPersistent().
- Don't use wfSetupSession() to make sure a session is active. Instead,
fetch a Session object as above and call ->persist().
- Don't use the other PHP session_*() functions either.
If you're implementing the UserLoadFromSession hook to bypass the usual
cookie-based session handling (e.g. OAuth, SSLClientAuthentication, or
CentralAuth's centralauthtoken), you'll now want to implement a subclass of
the new SessionProvider class and add it to $wgSessionProviders. OTOH, if
you're using UserLoadFromSession because you're trying to add additional
security checks (e.g. SecureSessions), you'll probably want to look into
the new SessionMetadata and SessionCheckInfo hooks. And if you're using
UserLoadFromSession because you're doing some hack around MediaWiki's
existing login form, you'll want to wait for AuthManager. The same goes for
UserSetCookies.
If you maintain an authentication-related extension that does a login of
some sort, you'll eventually want to be implementing one of AuthManager's
AuthenticationProviders. In particular, an extension implementing
AuthPlugin now will want to implement a PrimaryAuthenticationProvider in
the future. If you're using the hooks that mess with the login form, such
as UserCreateForm or UserLoginForm, you'll likely also be wanting to
implement an AuthenticationProvider of some sort. Other hooks called from
LoginForm might also need attention. You can see the work-in-progress patch
at https://gerrit.wikimedia.org/r/#/c/195297/.
A more detailed overview of the new classes is available at
https://www.mediawiki.org/wiki/User:Anomie/SessionManager_and_AuthManager.
If you have more questions, feel free to ask (email works particularly
well).
== For apps that log in ==
Apps shouldn't notice any changes from SessionManager. If you do see
problems related to session handling in 1.27.0-wmf.11, file a bug in
Phabricator and CC me (Anomie) and Gergő (Tgr).
Once AuthManager comes out, though, you'll need to use the new API
action=clientlogin and action=query&meta=authmanagerinfo which will provide
the information necessary to construct a fully-functional login UI. More
details will be provided once I finish writing the new code ;) but be aware
that there's the possibility that you'll have to redirect to a third-party
website in there (e.g. if a "login with your Google account" extension
happens to be installed).
Do not be tempted to have your app somehow use bot passwords, that's not a
supported use case for that feature. A normal (not owner-only) OAuth
consumer would be fine.
== For operators of third-party wikis ==
If you're using $wgSessionsInObjectCache = false (which was the default
before SessionManager), note that that configuration is no longer
supported. Make sure $wgSessionCacheType is set to something that will
function well in your environment.
Other than that, things should mostly just work on upgrade with both
SessionManager and (in the future) AuthManager. If you use authentication-
or session-related extensions that aren't deployed on WMF wikis, however,
you may want to check with the maintainers of those extensions.
If you're using the OAuth extension, you should also be aware that its
$wgMWOAuthGrantPermissions and $wgMWOAuthGrantPermissionGroups settings are
replaced by the new core settings $wgGrantPermissions and
$wgGrantPermissionGroups.
Please report bugs related to these changes in Phabricator, and CC me
(Anomie) and Gergő (Tgr).
--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hi,
I have been working on a project to improve the categorization of pictures
in the Upload to Commons Android app <
https://phabricator.wikimedia.org/T115101> as part of the Outreachy Dec '15
program, and I am happy to announce that Phase 1 of the project has been
implemented. :) The app should now suggest nearby categories when a picture
is uploaded.
Feedback on this feature would be greatly appreciated, so please feel free
to download the updated version of the app <
https://play.google.com/store/apps/details?id=fr.free.nrw.commons> and to
post feedback/issues on the GitHub page <
https://github.com/nicolas-raoul/apps-android-commons/issues/new>.
Please note that to use this new feature, you will need to have location
tagging enabled for your camera.
Thanks!
--
Regards,
Josephine
Excuse the forward, forgot to add this list...
---------- Forwarded message ----------
From: Joaquin Oltra Hernandez <jhernandez(a)wikimedia.org>
Date: Mon, Jan 11, 2016 at 7:59 PM
Subject: Video: Loading Barack Obama on 2G from Spain on a Nexus 5
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
Hi,
Here's an interesting video that we didn't get to show on the Developer
Summit.
This is me with my Nexus 5 (Android 6) on 2G in Spain loading
en.m.wikipedia.org/wiki/Barack
Obama <https://en.m.wikipedia.org/wiki/Barack%20Obama>
https://www.youtube.com/watch?v=W1w0EcuiUjo
Write your own conclusions, hopefully this will make us think.
If you can set your device from your country to 2G and try it out on an
incognito window (cold cache) and share the results, I would be very
interested to see if your experience is the same as mine.
Cheers,
Joaquin
In case you filter or don't subscribe to wikitech-l...
---------- Forwarded message ----------
From: *Florian Schmidt* <florian.schmidt.welzow(a)t-online.de>
Date: Tuesday, January 5, 2016
Subject: [Wikitech-l] [INFO] wfMsg*() was removed
To: "wikitech-l(a)lists.wikimedia.org" <wikitech-l(a)lists.wikimedia.org>
Hello readers of this list :)
tl;dr
MediaWiki's wfMsg*() functions were removed in MediaWiki 1.27.
Long version:
Maybe someone has already seen the work on the wfMsg*() deprecation task:
https://phabricator.wikimedia.org/T70750
<https://phabricator.wikimedia.org/T70750>
A little bit of background:
MediaWiki provides different functions to get a localised message string,
the one you normally choose is wfMessage, which creates a Message object.
However, there are other global functions (including wfMsg(),
wfMsgForContent(), wfMsgHtml() and so on), too, which was deprecated in
MediaWiki 1.18 (with deprecating notification in 1.21) in favour of
wfMessage. Unfortunately, a lot of (maybe unmaintained) extensions still
used the old deprecated functions to get a localised message.
After a huge amount of changes to these extensions (tracked in the linked
task), we now hope, that all usages of these functions are replaced by it's
modern wfMessage-counterpart, at least we did our best to find affected
extensions :)
Now, the time has come, that the change to mediawiki/core, which removes
the old deprecated functions[1], was merged. This notification is mostly
for people, who still use these functions, if you're sure you don't, you
can stop reading here :P
If you're a maintainer of an extension, please make sure, that you don't
use these wfMsg*() functions anymore (if your extension is hosted in
Wikimedia Gerrit, you probably mentioned a change named "Remove wfMsg*
calls", so we already did the work for you. If not, please take some
minutes to find out, how you can migrate to the new wfMessage function to
keep compatibility with newer MediaWiki releases. If you want to replace
your usage of wfMsg* functions your best friend (mostly) is the
documentation page[2], which describes appropriate replacements with the
actual message functions. However, it's possible, that your case of usage
isn't mentioned there, and if so, you could first try to find a better
approach using the Message object (returned by wfMessage()), which is
documented on doc.wikimedia.org <http://doc.wikimedia.org> [3], or, if you
really don't know, what to do, ask in #wikimedia-dev or on this mailing
list :) I think you'll get help as soon as possible.
I hope this answers all questions, if not, feel free to answer this e-mail
:)
Best,
Florian
[1] https://gerrit.wikimedia.org/r/#/c/262333
<https://gerrit.wikimedia.org/r/#/c/262333>
[2] https://www.mediawiki.org/wiki/Manual:Messages_API#Help_
with_replacing_deprecated_wfMsg.2A_functions
<
https://www.mediawiki.org/wiki/Manual:Messages_API#Help_with_replacing_depr…
>
[3] https://doc.wikimedia.org/mediawiki-core/master/php/
html/classMessage.html#
<https://doc.wikimedia.org/mediawiki-core/master/php/html/classMessage.html>
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org <javascript:;>
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hi Wiktor,
Did Google specify which portion(s) of their regulations your app was
violating?
If you think the issue might have involved CC BY-SA, then you may want to
look at Wikipedia:Reusing Wikipedia Content
<https://en.wikipedia.org/wiki/Wikipedia:Reusing_Wikipedia_content>. That
policy may help you determine if your app complied with the Creative
Commons license restrictions. (And for more information about attribution,
you can consult this page on the CC wiki:
https://wiki.creativecommons.org/wiki/Best_practices_for_attribution.)
Best,
Charles M. Roslof
Intellectual Property & Internet Law Fellow
Wikimedia Foundation
croslof(a)wikimedia.org
(415) 839-6885
NOTICE: *This message might contain confidential or legally privileged
information. If you have received this message by accident, please delete
it and let us know about the mistake. For legal reasons, I may only serve
as a legal fellow for the Wikimedia Foundation. This means I may not give
legal advice to or serve as a lawyer for community members, volunteers, or
staff members in their personal* *capacity.* For more on what this means,
please see our legal disclaimer
<https://meta.wikimedia.org/wiki/Wikimedia_Legal_Disclaimer>.
On Sun, Jan 3, 2016 at 1:32 PM, Wiktor Warchałowski <
w.warchalowski(a)gmail.com> wrote:
> Dear Sir or Madam,
>
> My name is Wiktor Warchałowski and I am software developer from Poland. In
> the previous month, together with my friends from the University of Science
> and Technology in Krakow, we wrote the application for Android called
> "WikiBuster".
>
> The idea is quite simple - user has to move from a random article on
> Wikipedia (Polish or English - depending on his preference) to another
> article using only hyperlinks to pages. The application works like a
> browser, except that WIkiBuster app collects the number of clicks between
> first and last article.
>
> The idea that guided us was to promote knowledge and Wikipedia. At the
> beginning we have promoted our app among friends, to get feedback, what do
> they think about this idea and form of entertainment combined with
> education.
>
> Unfortunately, some time ago Google Play removed our app from the store
> due to violations of the Google regulations. I have wrote to
> polish Wikipedia information team and sent them our app. They found it very
> interesting.
>
> They suppose that Google removed our app because of license CC BY-SA
> 4.0. They told us to write to Wikimedia Foundation and Wikimedia Mobile
> engineering department. Can you help us to adapt the application to the
> license terms?
>
> We can send you our application in .app format if needed.
>
> Yours faithfully,
> Wiktor
>
>
>
>
>