Hi,
The report covering Wikimedia engineering activities in July 2014 is now
available.
Wiki version:
https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/July
Blog version:
https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/
We're also proposing a shorter version of this report focusing on priority
goals for this quarter:
https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/July/summa…
Below is the HTML text of the report's summary.
As always, feedback is appreciated on the usefulness of the report and its
summary, and on how to improve them.
------------------------------------------------------------------
Major news in July include:
- a recap of how the Operations team collaborated with the RIPE NCC
<https://blog.wikimedia.org/2014/07/09/how-ripe-atlas-helped-wikipedia-users/>
to
measure the delivery of Wikimedia sites to users in Asia and elsewhere;
- an analysis of the impact of the San Francisco data center
<https://blog.wikimedia.org/2014/07/11/making-wikimedia-sites-faster/> on
the speed of Wikimedia sites;
- the launch of the new native Wikipedia app for iOS
<https://blog.wikimedia.org/2014/07/31/official-wikipedia-app-available-on-ios-and-android/>
;
- a first look at the content translation tool
<https://blog.wikimedia.org/2014/07/16/first-look-at-the-content-translation-tool/>
.
HHVM <https://www.mediawiki.org/wiki/HHVM>
HHVM (HipHop Virtual Machine
<https://en.wikipedia.org/wiki/HipHop_Virtual_Machine>) is aimed to improve
the speed of Wikimedia sites. The Beta cluster
<https://www.mediawiki.org/wiki/Beta_cluster> (the testing environment that
best simulates our sites) is now running HHVM. The latest MediaWiki-Vagrant
<https://www.mediawiki.org/wiki/MediaWiki-Vagrant> and Labs-vagrant
<https://wikitech.wikimedia.org/wiki/Labs-vagrant> (virtual machine
environments that make it easier for developers to apply their code to
Wikimedia sites) use HHVM by default.
Mobile Apps <https://www.mediawiki.org/wiki/Wikimedia_Apps>
In July, the Mobile Apps team launched the new native iOS Wikipedia app,
following on from the successful launch of the Android app in June. The app
has the same features as the Android app, including the ability to edit
both anonymously and logged in, saved pages for offline reading, and your
recently visited pages. The iOS app also contains an onboarding screen
which is displayed the first time the app is launched, asking users to sign
up. An update to the Android app was released, containing the Android
version of the onboarding screen, as well as a a night mode for reading in
dark environments, a font size selector, and a references display that
makes browsing references easier. Next month, the team plans to continue
improvements to page styling, and begin designing a dialogue that displays
the first time a user taps edit to help them make their edit successfully.
Mobile Web <https://www.mediawiki.org/wiki/Mobile_web_projects>
This month, the team continued to focus on wrapping up the collaboration
with the Editing team to bring VisualEditor to tablet users on the mobile
site. We also began working to design and prototype our first new Wikidata
contribution stream, which we will build and test with users on the beta
site in the coming month.
Flow <https://www.mediawiki.org/wiki/Flow>
In July, the Flow team built the ability for users to subscribe to
individual Flow discussions, instead of following an entire page of
conversations. Subscribing to an individual thread is automatic for users
who create or reply to the thread, and users can choose to subscribe (or
unsubscribe) by clicking a star icon in the conversation's header box.
Users who are subscribed to a thread receive notifications about any
replies or activity in that thread. To support the new
subscription/notification system, the team created a new namespace, Topic,
which is the new "permalink" URL for discussion threads; when a user clicks
on a notification, the target link will be the Topic page, with the new
messages highlighted with a color. The team is currently building a new
read/unread state for Flow notifications, to help users keep track of the
active discussion topics that they're subscribed to.
VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>
In July, the team working on VisualEditor converged the mobile and desktop
designs, made it possible to see and edit HTML comments, improved access to
re-using citations, and fixed over 120 bugs and tickets
<https://bugzilla.wikimedia.org/buglist.cgi?list_id=333021&order=priority%2Cbug_severity&product=VisualEditor&query_format=advanced&resolution=FIXED&target_milestone=VE-deploy-2014-07-03&target_milestone=VE-deploy-2014-07-10&target_milestone=VE-deploy-2014-07-17&target_milestone=VE-deploy-2014-07-24&target_milestone=VE-deploy-2014-07-31>.
The team also expanded its scope to cover all MediaWiki editing tools as
well, as the new Editing Team.
The new design is possible due to the significant progress made in
cross-platform support in the interface code. This now provides
responsively-sized windows that can work on desktop, tablet and phone with
the same code. HTML comments are occasionally used to alert editors to
contentious issues without disrupting articles for readers. Making them
prominently visible avoids editors accidentally stepping over expected
limits. Re-using citations with its simple dialog is now available in the
toolbar so that it is easier for users to find.
Other improvements include an array of performance fixes targeted at
helping mobile users especially. We fixed several minor instances where
VisualEditor would corrupt the page. We also installed better monitoring of
corruptions if they occur. The mobile version of VisualEditor, currently
available for beta testers, moved towards stable release. We fixed some
bugs and editing issues, and improving loading performance. Our work to
support languages made some significant gains, nearing the completion of a
major task to support IME <https://en.wikipedia.org/wiki/Input_method> users.
The work to support Internet Explorer uncovered some more issues as well as
fixes.
SUL finalization <https://www.mediawiki.org/wiki/SUL_finalisation>
In July, the SUL finalisation team worked on developing features to ease
the workload that the finalisation will place on the community, and to
minimise the impact on those users that are affected. A feature is being
developed that allows users to log in with their pre-finalisation
credentials, so that everyone who is affected is still able to access their
account; this feature is mostly complete from a back-end engineering
standpoint but now needs design and product refinement, and will hopefully
be completed by late August. A feature to globally rename users in a manner
that does not create clashing accounts was completed and deployed. A
feature is being developed to allow accounts to be globally merged, so that
clashing local-only accounts that were globalised by the finalisation can
be consolidated into a single global account; this feature is in the early
stages of implementation and no estimate is possible at this time. A
feature is being developed to allow local-only account holders to request
rename and globalisation before the finalisation, and also feeds these
rename requests to the appropriate community processes in a manner that
reduces the workload of community; this feature is in the design phase, and
will likely be ready for implementation in early August.
Phabricator migration
<https://www.mediawiki.org/wiki/Phabricator/Migration>
Phabricator's "Legalpad" application (a tool to manage trusted users
<http://fab.wmflabs.org/T364>) was set up on a separate server that
provides provides Single-User Login authentication with wiki credentials.
We implemented the ability to restrict access to tasks in a certain project
<http://fab.wmflabs.org/T95> and worked on initial migration code to import
data from Bugzilla reports into Phabricator tasks. We also set up a data
backup system for Phabricator, and upgraded the dedicated Phabricator
server to Ubuntu Trusty. A more detailedsummary email
<http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077684.html> about
the status of the Phabricator migration was sent to wikitech-l.
MediaWiki core front-end libraries
<https://www.mediawiki.org/wiki/Requests_for_comment/Redo_skin_framework>
In July, the Request for comment for refactoring MediaWiki's skin system
(which handles the appearance of wiki sites) was re-written and discussed
with members of the community and staff. Work on the proposed system is
scheduled to begin in August, alongside creating an Agora
<https://www.mediawiki.org/wiki/Wikimedia_Foundation_Design> theme for, and
server-side version of, OOjs UI <https://www.mediawiki.org/wiki/OOjs_UI>, a
toolkit used to compose complex widgets. In addition to the RfC work, a
well-attended meeting was held for teams using or considering using OOjs
UI, including Editing, Multimedia and Growth. From that meeting, several
issues were identified as blockers to increased acceptance of the toolkit.
The most prominent blocker is the lack of an Agora theme for OOjs UI at
this time. Creating this theme has thus been prioritized and will be
completed as soon as possible. The Design team
<https://www.mediawiki.org/wiki/Design> has committed to delivering
necessary assets by mid-August. Discussion about changes to OOjs UI also
surfaced the desire to be able to create widgets on the server and then
bind to them on the client (a feature proposed as part of the skinning
RfC). This functionality is thus now planned to be implemented in OOjs UI
before the skin refactoring begins.
Metrics and dashboards standardization
<https://www.mediawiki.org/wiki/Analytics/Research_and_Data>
This month we completed the documentation for the Active Editor Model
<https://meta.wikimedia.org/wiki/Research:Modeling_monthly_active_editors>,
a set of metrics for observing sub-population trends and setting product
team goals. We also engaged in further work on the new page views
definition. An interim solution for Limited-duration Unique Client
Identifiers (LUCIDs) was also developed and passed to the Analytics
Engineering team for review.
We analyzed trends in mobile readership and contributions, with a
particular focus on the tablet switchover and the release of the native
Android app. We found
<https://meta.wikimedia.org/wiki/Research:Mobile_editor_engagement/Editor_activation>
that
in the first half of 2014 mobile surpassed desktop in the rate at which new
registered users become first-time editors and first-time active editors in
many major projects, including the English Wikipedia. An update on mobile
trends will be presented at the upcoming Monthly Metrics meeting on July 31.
Content API <https://www.mediawiki.org/wiki/Services>
The brand new Services <https://www.mediawiki.org/wiki/Services> group
started design and prototyping work on the storage service
<https://www.mediawiki.org/wiki/Requests_for_comment/Storage_service> (see
code <https://github.com/gwicke/rashomon>) and REST API
<https://www.mediawiki.org/wiki/Requests_for_comment/Content_API> (see code
<https://github.com/gwicke/restface>). The storage service now has early
support for bucket creation and multiple bucket types. We decided to
configure the storage service as a back-end for the REST API server. This
means that all requests will be sent to the REST API, which will then route
them to the appropriate storage service without network overhead. This
design lets us keep the storage service buckets very general by adding
entry point specific logic in front-end handlers. The interface is still
well-defined in terms of HTTP requests, so it remains straightforward to
run the storage service as a separate process. We refined the bucket design
to allow us to add features very similar to Amazon DynamoDB
<https://en.wikipedia.org/wiki/Amazon_DynamoDB> in a future iteration.
There is also an early design for light-weight HTTP transaction support.
--
Guillaume Paumier
Technical Communications Manager — Wikimedia Foundation