Hi,
The report covering Wikimedia engineering activities in February 2014 is
now available.
Wiki version:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/February
Blog version:
https://blog.wikimedia.org/2014/03/15/engineering-report-february-2014/
We're also proposing a shorter, simpler and translatable version of this
report that does not assume specialized technical knowledge:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/February/s…
Below is the HTML text of the report.
As always, feedback is appreciated on the usefulness of the report and its
summary, and on how to improve them.
------------------------------------------------------------------
Major news in February include:
- a call for volunteers to test the upcoming multimedia
viewer<https://blog.wikimedia.org/2014/02/27/help-test-media-viewer/>
;
- improvements to VisualEditor’s media and template editors;
- the launch of the Flow discussion system on two pilot talk pages on
the English Wikipedia;
- the launch of guided tours to 31 more language versions of Wikipedia,
including all of the top 10 projects by number of page views;
- improvements to the tools and process used to deploy code to Wikimedia
production sites;
- the release of the first archive of the entire English Wikipedia with
thumbnails<http://lists.wikimedia.org/pipermail/offline-l/2014-March/001…ml>,
for offline use.
*Note: We’re also providing a shorter, simpler and translatable version of
this report
<https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/February/summary>
that does not assume specialized technical knowledge.*
Engineering metrics in February:
- 149 unique committers contributed patchsets of code to MediaWiki.
- The total number of unresolved
commits<https://gerrit.wikimedia.org/r/#q,status:open+project:%255Emedia…
from around 1320 to about 1453.
- About 22 shell requests
<https://www.mediawiki.org/wiki/Shell_requests>were processed.
Contents
- 1
Personnel<https://blog.wikimedia.org/2014/03/15/engineering-report-febru…
- 1.1 Work with
us<https://blog.wikimedia.org/2014/03/15/engineering-report-february-201…
- 1.2
Announcements<https://blog.wikimedia.org/2014/03/15/engineering-report-f…
- 2 Technical
Operations<https://blog.wikimedia.org/2014/03/15/engineering-report-febr…
- 3 Features
Engineering<https://blog.wikimedia.org/2014/03/15/engineering-report-feb…
- 3.1 Editor retention: Editing
tools<https://blog.wikimedia.org/2014/03/15/engineering-report-february-…
- 3.2 Core
Features<https://blog.wikimedia.org/2014/03/15/engineering-report-februa…
- 3.3
Growth<https://blog.wikimedia.org/2014/03/15/engineering-report-february…
- 3.4
Support<https://blog.wikimedia.org/2014/03/15/engineering-report-februar…
- 4
Mobile<https://blog.wikimedia.org/2014/03/15/engineering-report-february…
- 5 Language
Engineering<https://blog.wikimedia.org/2014/03/15/engineering-report-feb…
- 6 Platform
Engineering<https://blog.wikimedia.org/2014/03/15/engineering-report-feb…
- 6.1 MediaWiki
Core<https://blog.wikimedia.org/2014/03/15/engineering-report-february-2…
- 6.2 Quality
assurance<https://blog.wikimedia.org/2014/03/15/engineering-report-febru…
- 6.3
Multimedia<https://blog.wikimedia.org/2014/03/15/engineering-report-febr…
- 6.4 Engineering Community
Team<https://blog.wikimedia.org/2014/03/15/engineering-report-february-2…
- 7
Analytics<https://blog.wikimedia.org/2014/03/15/engineering-report-febru…
- 8
Kiwix<https://blog.wikimedia.org/2014/03/15/engineering-report-february-…
- 9
Wikidata<https://blog.wikimedia.org/2014/03/15/engineering-report-februa…
- 10
Future<https://blog.wikimedia.org/2014/03/15/engineering-report-february…
Personnel Work with us <https://wikimediafoundation.org/wiki/Work_with_us>
Are you looking to work for Wikimedia? We have a lot of hiring coming up,
and we really love talking to active community members about these roles.
- VP of
Engineering<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&…
- Software Engineer –
Growth<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&c…
- Software Engineer – VisualEditor
(
Features)<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&am…
- Software Engineer – Language
Engineering<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&…
- Software Engineer- Mobile
(
Frontend)<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&am…
- Software Engineer – Mobile (Android
Apps)<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs…
- Automation
Engineer<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&…
- Release
Engineer<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&…
- Director of Community Engagement
(
Product)<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&…
- Product Manager –
Multimedia<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&a…
- Operations Security
Engineer<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&…
Announcements
- Leila Zia joined the Analytics team as Research Scientist
(
announcement<http://lists.wikimedia.org/pipermail/analytics/2014-Februar…
).
- Faidon Liambotis was promoted to Principal Operations Engineer (
announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2014-Februa…
).
- YuFei Liu joined the UX Design team as Visual Design Intern (
announcement<http://lists.wikimedia.org/pipermail/design/2014-February/0…
).
- Following changes in the Language engineering team, Amir Aharoni is
now the Acting Product Manager, and Runa Bhattacharjee the ScrumMaster (
announcement<http://lists.wikimedia.org/pipermail/mediawiki-i18n/2014-Fe…
).
Technical Operations
*Datacenter RFP <https://wikimediafoundation.org/wiki/RFP/2013_Datacenter>*
Final negotiations have completed with the 3 remaining data center bids in
February, and the Wikimedia Operations team will make a decision in the
first week of March. Expect a public announcement soon.
*Wikimedia Labs*
Labs metrics in February:
- Number of projects: 129
- Number of instances: 458
- Amount of RAM in use (in MBs): 1,812,992
- Amount of allocated storage (in GBs): 24,540
- Number of virtual CPUs in use: 906
- Number of users: 2,714
The Wikimedia Labs infrastructure in the *eqiad* data center has been
deployed with the OpenStack
Havana<https://www.openstack.org/software/havana/>releaseelease, and testing
completed in February. Labs users will have 2 weeks to
migrate their own projects & instances starting in March. During the last
two weeks of March, the Wikimedia Operations team will handle the transfer
of the remaining instances that have not been migrated by users themselves.
*ulsfo redeployment*
During a short deployment of our West Coast data center *ulsfo* in October
2013 several reliability problems were found with some of our network
service providers, which forced us to take this site out of service until
they could be resolved. We have worked since to improve reliability and
increase redundancy of network transit and transport to this site. As of
the week of February 3rd *ulsfo* is in full production usage again, and is
now serving traffic for the US west coast, Oceania and large parts of Asia.
A blog post is being prepared describing the improvements in user perceived
site performance.
*eqiad data center capacity expansion*
The Wikimedia Foundation has expanded the capacity of its main data center
site *eqiad* in Ashburn, Virginia by 33%. A fourth row of racks has been
added, and all power & networking infrastructure has been installed and
configured in February. The added rack space is available for new equipment
as of February 24th.
Features
Engineering<https://www.mediawiki.org/wiki/Wikimedia_Features_engineerin…
Editor
retention: Editing tools
*VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>*
In February, the VisualEditor team continued their work on improving the
stability and performance of the system, and added some new features and
simplifications. Media item editing is now much richer, allowing the
setting of position, alt text, size (or setting as default size) and type
for most kinds of media item. When adding links, redirects and
disambiguation pages are now highlighted to help editors select the right
link, and changing the format or style of some text was tweaked to make
editing clearer and more obvious. Adding and editing template usages is now
a little smoother, auto-focussing on parameters and making them clearer to
use. Page settings have expanded to set redirects, page indexing and new
section edit link options. The extensive work to make insertion of
“citation” references based on templates quick, obvious and simple neared
completion. The deployed version of the code was updated four times in the
regular releases
(
1.23-wmf13<https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf13#VisualEdi…or>,
1.23-wmf14<https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf14#VisualEdi…or>,
1.23-wmf15<https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf15#VisualEdi…
1.23-wmf16<https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf16#VisualEdi…
).
*Parsoid <https://www.mediawiki.org/wiki/Parsoid>*
In February, the Parsoid team continued with bug fixes and improved image
support. See the deployment
page<https://www.mediawiki.org/wiki/Parsoid/Deployments>for a summary
of deployments and fixed bugs in February.
Part of the team has continued to mentor two Outreach Program for
Women<https://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Rou…
interns. This program ends mid-March. Others are mentoring a group of
students in a Facebook Open
Academy<https://www.mediawiki.org/wiki/Facebook_Open_Academy>project
to build a Cassandra storage back-end for the Parsoid round-trip
test server.
We have a first version of a Debian package for Parsoid ready. This package
is yet to find a home base (repository) from which it can be installed.
This will soon make the installation of Parsoid as easy as apt-get install
parsoid.
Core Features
*Flow <https://www.mediawiki.org/wiki/Flow/Project_information>*
This month, Flow was launched on the talk pages of two English Wikipedia
WikiProjects that volunteered to be a part of the first trial, WikiProject
Breakfast <https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Breakfast>and
WikiProject
Hampshire <https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Hampshire>.
We’ve continued to iterate on the front-end design of the discussion system
based on user feedback, releasing a new visual treatment during the trial
and starting work on a front-end rewrite for better cross-browser and
mobile compatibility (to be released sometime in March). We also spent time
making sure Flow integrates better with vital MediaWiki tools and processes
(e.g., suppression and checkuser) and improving the handling of permalink
URLs.
Growth
*Growth <https://www.mediawiki.org/wiki/Growth>*
<https://commons.wikimedia.org/wiki/File:Growth_Quarterly_Review_%28February_2014%29.pdf>
<https://commons.wikimedia.org/wiki/File:Growth_Quarterly_Review_%28February_2014%29.pdf>
Slides of the quarterly review.
In February, the Growth team first focused on releasing the new Wikipedia
onboarding
experience<https://www.mediawiki.org/wiki/Onboarding_new_Wikipedians>…
additional projects. The
GettingStarted <https://www.mediawiki.org/wiki/Extension:GettingStarted>extension
was deployed to 30 Wikipedias, including all of the top 10
projects by number of page views. This marks the first time its task
suggestions and guided tours were available outside English projects. The
GuidedTour <https://www.mediawiki.org/wiki/Extension:GuidedTour> extension
was also deployed to those projects (as a dependency of GettingStarted), as
well as the Czech Wikipedia and
se.wikimedia.org. Late in the month, the
team also presented its work at its first Quarterly Review of the 2014
calendar year (see slides and
minutes<https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/…
).
Support
*Wikipedia Education Program
<https://www.mediawiki.org/wiki/Wikipedia_Education_Program>*
For the first half of the month, we focused on the current Education
Program extension. We fixed many old and new bugs—including a few remaining
database-related problems—and improved the UI for editing courses. Also,
two Facebook Open
Academy<https://www.mediawiki.org/wiki/Facebook_Open_Academy>students
started work on new notifications for the extension. In
mid-February the team shifted our focus to creating new software for many
kinds of collaborative editing, including, but not limited to, Education
Program courses. The first phase of this work, called editor
campaigns<https://www.mediawiki.org/wiki/Editor_campaigns>ns>,
is being carried out with the Growth
<https://www.mediawiki.org/wiki/Growth>team.
Mobile <https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering>
*Wikimedia Apps <https://www.mediawiki.org/wiki/Wikimedia_Apps>*
We’ve worked primarily on enabling wikitext editing, specifically enabling
logged-out editing, logged-in editing, logging in and creating accounts.
*Mobile web projects <https://www.mediawiki.org/wiki/Mobile_web_projects>*
We’ve been working on bringing VisualEditor to tablets (currently in
alpha). This is a requirement for redirecting tablets to mobile later on.
Specifically, we’ve been working on enabling inspectors, especially the
link inspector. We’ve also been fixing a variety of bugs to ensure that the
basic editing functionality works as expected.
*Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>*
During the last month, the team added zero-rating for HTTPS for select
carriers in cooperation with the Operations team. In collaboration with the
Mobile Apps team, we integrated Wikipedia Zero into the forthcoming
rebooted versions of the Android and iOS apps, including API and
client-side code for zero-rating detection. We updated the legacy Firefox
OS app with bugfixes from January (make spinner background opaque, remove
mozmarket.js legacy JS); we also prepared other bugfixes for that app (keep
last page browsed on low memory crash, avoid text overlaying <select>
dropdwon, ensure ‘X’ clicks stop processing and not send user to Main
Page). Discussion with the Operations team and Platform Engineering
continued on the ideal portal hosting approach concurrent with sprint
planning; portal work is probably deferred until the hosting strategy is
formalized. The team also started work on the core API to allow dynamic
category pages based on search terms, as well as continuing the discussion
on core ResourceLoader features, in support of a proof of concept HTML5
webapp riding atop MobileFrontend. We also started a patch to make
contributory features (not just banners and rewritten URLs) present for
Wikipedia Zero users on carriers supporting HTTPS zero-rating. Last but not
least, Yuri Astrakhan performed extensive analytics work on pageviews and
page bandwidth consumption for gzip-capable Wikipedia Zero clients across
all Wikipedia Zero-scoped partner pageviews; Yuri also conducted additional
analytics work on SMS/USSD data.
*Wikipedia Zero (partnerships)*
In February, we launched Wikipedia Zero with MTN South Africa (Opera Mini
browser only). MTN South Africa responded directly to the kids of
Sinenjongo High School with an open letter to the students and the youth of
South Africa. They said they agree that Wikipedia could give a boost to
their education system, and that offering Wikipedia Zero is a small thing
that could change everything (see
video<http://www.youtube.com/watch?v=sc2lVRtWEvM>on YouTube).We
also launched Wikipedia Zero with Safaricom, the largest operator in Kenya.
We now have three partners in Kenya, covering 90% of all mobile
subscribers. South Africa is our 23rd country to launch, and Safaricom is
our 27th operator partner.The Mobile Partnerships team attended Mobile
World Congress in Barcelona, where we met with existing operator partners,
prospective partners and tech companies who want to support the mission. At
the conference, our Wikipedia Text pilot with Airtel Kenya and the Praekelt
Foundation was nominated as a finalist for the GSMA Global Mobile awards in
the education category.
Language
Engineering<https://www.mediawiki.org/wiki/Wikimedia_Language_engineerin…
*Language tools <https://www.mediawiki.org/wiki/Language_tools>*
UniversalLanguageSelector<https://www.mediawiki.org/wiki/UniversalLangua…
re-enabled with webfonts disabled by default. Research is ongoing to
see whether they can and should be re-enabled by default at least for some
languages.
More convenient shortcuts were added by Niklas Laxström to the Translate
extension.
Kartik Mistry and Amir Aharoni are working on stabilizing the browser tests
for all the language extensions and on setting up more robust online
staging sites.
*Milkshake <https://www.mediawiki.org/wiki/Milkshake>*
Several bugs were fixed in jquery.webfonts.
*Language Engineering Communications and Outreach
<https://www.mediawiki.org/wiki/Language_engineering_communications_and_outreach>*
Runa Bhattacharjee is setting up a Test Case Management System, to
facilitate manual testing inside the team and helping volunteer translators
test new versions of language tools and report the results.
*Content translation <https://www.mediawiki.org/wiki/Content_translation>*
The prototype ContentTranslation server was created in Node.js, mostly by
Santhosh Thottingal and David Chan. The server will be responsible for
syncing the translations between all the languages, storing translated
parallel texts (using Redis) and retrieving caching the results of language
tools queries (machine translation, translation memory, dictionaries,
segmentation, etc.). Some front-end components for the translation
interface were made, mostly by Sucheta Goshal and Amir Aharoni.
Platform
Engineering<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineerin…
MediaWiki
Core
*HipHop deployment <https://www.mediawiki.org/wiki/HipHop_deployment>*
Work is starting back up on this project, with the goal of having at least
one production service running on HipHop by the end of the quarter. Tim
Starling is working with the HHVM
upstream<https://github.com/facebook/hhvm/issues/1697>to finish off a
compatibility layer for running Zend extensions (
ext_zend_compat) under HipHop, with the goal of using it for our Lua
module. Ori Livneh is working on packaging and deployment issues, as well
as generally wrangling the overall development effort. Aaron Schulz is
starting to investigate what is needed for wmferrors support.
*Release & QA
<https://www.mediawiki.org/wiki/Wikimedia_Release_and_QA_Team>*
<https://commons.wikimedia.org/wiki/File:Wikimedia_development_and_deployment_flowchart.png>
<https://commons.wikimedia.org/wiki/File:Wikimedia_development_and_deployment_flowchart.png>
Wikimedia development and deployment flowchart
The Release and QA team had their latest quarterly
review<https://www.mediawiki.org/wiki/Wikimedia_Release_and_QA_Team/Quar…
February 13. Highlights from the meeting include:
- We will be hiring two new positions (a QA Automation Engineer and a
Test Infrastructure Engineer).
- We will process through all pain points from the Development and
Deployment process
review<https://www.mediawiki.org/wiki/Development_and_Deployment_Process…
.
- We will continue performing incremental improvements to the current
deployment script (known as “scap”) to better inform future deployment
tooling work.
- We will create a way for tests to create fake/stub data (for use in
throw-away/one-off test instances).
- We will make it so our browser tests are more accurate cross testing
and production environments.
Notable progress on things with visuals includes an updated Development and
deployment flowchart (opposite), as well as an auto-generated
version<https://doc.wikimedia.org/mw-tools-releng/html/devdeployflow/ind…
.
*Admin tools development
<https://www.mediawiki.org/wiki/Admin_tools_development>*
While this workstream is still officially on hold, the related Global
CSS/JS<https://www.mediawiki.org/wiki/Extension:GlobalCssJs>extension
to provide per-user global modules was deployed to beta labs for
testing. Additionally, patches were contributed by volunteer developers.
*Search <https://www.mediawiki.org/wiki/Search>*
This month, almost all LuceneSearch and MWSearch bugs have either been
closed as problems that are fixed in CirrusSearch, or moved to the
CirrusSearch component. We then prioritized all CirrusSearch bugs. After
clearing out any remaining high priority issues, engineering work for an
update to the design of the search results page is due to commence on March
10.
*Wikimania Scholarships app
<https://www.mediawiki.org/wiki/Wikimania_Scholarships_app>*
The application automatically transitioned from the active scholarship
collection period to the review-only period on 2014-02-17. No major issues
were reported for February. The back-end features of the application were
demoed for the IEG <https://meta.wikimedia.org/wiki/Grants:IEG> team as
part of their information gathering process for implementing a more
structured review tool for grants.
*Deployment tooling <https://www.mediawiki.org/wiki/Deployment_tooling>*
The month of February saw a lot of work on WMF deployment tooling.
To see a real life example of what it looks like to deploy code on the WMF
server cluster, watch this screencast
<https://asciinema.org/a/7798>created by Bryan Davis. That shows you
what the person deploying the code
sees when doing a localization (translations) update. A deployment that
includes new changes to the code (e.g. MediaWiki and extensions) on the
servers would be different.
The suite of tools that make up the current MediaWiki deployment tooling is
continuing to be updated and rewritten in Python. You can see the work of
this in the repository’s
history<https://git.wikimedia.org/log/mediawiki%2Ftools%2Fscap/HEAD>
.
The updated Development and Deployment
Process<https://www.mediawiki.org/wiki/Development_and_Deployment_Proces…
is now created using
Blockdiag <http://blockdiag.com>, a Python library for converting text into
flow charts. You can see the current draft in the newly-minted Release
Engineering
repository<https://git.wikimedia.org/tree/mediawiki%2Ftools%2Freleng>
.
There is now a matrix showing the requirements for deployment
tooling<https://www.mediawiki.org/wiki/Deployment_tooling/Notes/Deployme…
3 projects (MediaWiki, Parsoid (and related), and ElasticSearch (and
related)). This is not a fixed document and will grow/change as more is
learned.
*Security auditing and response
<https://www.mediawiki.org/wiki/Security_auditing_and_response>*
MediaWiki 1.22.3, 1.21.6, and 1.19.12 security updates were released. We
started a review of the Hadoop infrastructure and the Popups extension.
Quality assurance
*Quality Assurance <https://www.mediawiki.org/wiki/Quality_Assurance>*
In February, we updated our 3rd-party Jenkins instance to use Jenkins job
builder configuration rather than Jenkins templates. Now our 3rd-party
Jenkins builds matches the WMF Jenkins build scheme, giving us maximum
flexibility for when and how these jobs are run in the future. Also, we
laid the groundwork for several significant new test features to be
announced in the near future.
*Beta cluster <https://www.mediawiki.org/wiki/Beta_cluster>*
Not much happened on the beta cluster beside the usual maintenance and the
platform being used to detect nasty bugs before they land on the production
cluster. It is being used successfully for staging various features,
bugfixes and extensions as well as for browser tests tracking regressions.
Next month will see the beta cluster migrating from the pmtpa datacenter to
the eqiad datacenter.
*Continuous integration
<https://www.mediawiki.org/wiki/Continuous_integration>*
Two instances in labs have been added as Jenkins slaves. They are equipped
with tox and pip to let us tests python software while fetching
dependencies from pypi (bug
44443<https://bugzilla.wikimedia.org/show_bug.cgi?id=44443>
).
Nik Everett made the CirrusSearch browsertests runnable on a labs instance
which has elastic search. The job is now triggered from Gerrit and being
improved.
The experimental Meetbot instance setup by Antoine back in November has
been overhauled and is now maintained by the community in the tools-labs
project (thank you Tim Landscheidt).
Several Debian packages are now build automatically via Jenkins thanks to
an effort by Carl Fürstenberg
https://integration.wikimedia.org/ci/view/Ops-DebGlue/ . It helped
packaging Parsoid among others.
*Browser testing
<https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing>*
Our test coverage of MediaWiki extensions continues to prove itself. In
February, using the automated browser tests running against beta labs and
test2wiki, we found and fixed several critical errors that would have
disrupted production wikis severely if they had been released.
Multimedia
*Multimedia <https://www.mediawiki.org/wiki/Multimedia>*
<https://www.mediawiki.org/w/index.php?title=File:Media_Viewer_v0.2_Slides_-_WMF_Metrics_Meeting_%2803-06-2014%29.pdf&page=8>
<https://commons.wikimedia.org/wiki/File:Media_Viewer_v0.2_Slides_-_WMF_Metrics_Meeting_%2803-06-2014%29.pdf?page=8>
Presentation slides about Media Viewer
In February, the multimedia team continued to focus on Media Viewer
v0.2<https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer>er>,
getting it ready for a wider release next
quarter<https://blog.wikimedia.org/2014/02/27/help-test-media-viewer/>r/>.
Gilles Dubuc, Mark Holmquist, Gergő Tisza and Aaron Arcos released a
variety of new features, such as:
permissions<https://wikimedia.mingle.thoughtworks.com/projects/multimedi…18>,
file usage <https://mingle.corp.wikimedia.org/projects/multimedia/cards/44>,
pre-loading of
images<https://mingle.corp.wikimedia.org/projects/multimedia/cards/155&g…55>,
previews during
load<https://mingle.corp.wikimedia.org/projects/multimedia/cards/146>…
an improved full-screen
experience <https://mingle.corp.wikimedia.org/projects/multimedia/cards/111>.
We also started development on a better ‘Use this file’ panel, including
share<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/card…47>,
embed<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/card…
download<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/c…atures.
Pau Giner designed this panel, as well as a new Zoom
feature<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/ca…
next quarter’s v0.3 version of Media Viewer. We invite you to test
the latest version <https://www.mediawiki.org/wiki/Lightbox_demo> (see
the testing
tips<https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer#How_ca…)
and share your
feedback<https://www.mediawiki.org/wiki/Talk:Multimedia/About_Media_View…
.
Fabrice Florin managed product development for Media Viewer and prepared
the release
plan<https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan…
a gradual deployment of Media Viewer out of beta in coming months,
based on the team’s latest development
goals<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/card…s=Must+have%2CShould+have%2CCould+have&tab=Current+release>.
We also hosted an IRC
chat<https://meta.wikimedia.org/wiki/IRC_office_hours/Office_hours_2014-…
discuss Media Viewer with the rest of the community and plan our next
steps together. Lastly, the video
RfC<https://commons.wikimedia.org/wiki/Commons:Requests_for_comment/MP4_…
started last month was closed with a community recommendation to not
support the proprietary MP4 video format on our sites; as a result, we will
only support open video formats like WebM and Ogg in the next version
(
v0.3)<https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer#Next_…
Media Viewer. For more updates, we invite you to join the multimedia
mailing list <https://lists.wikimedia.org/mailman/listinfo/multimedia>.
Engineering Community
Team<https://www.mediawiki.org/wiki/Engineering_Community_Team>
*Bug management <https://www.mediawiki.org/wiki/Bug_management>*
Bugzilla got upgraded from version 4.2.7 to
4.4.1<https://bugzilla.wikimedia.org/show_bug.cgi?id=49597>97>,
which fixed numerous
bugs<https://bugzilla.wikimedia.org/buglist.cgi?bug_id=47013,42850,32504…96>.
Daniel Zahn puppetized
Bugzilla<https://bugzilla.wikimedia.org/show_bug.cgi?id=51036>and
(together with Sean Pringle) moved Wikimedia Bugzilla to a new
server <https://wikitech.wikimedia.org/wiki/Zirconium>. Bugzilla now
displays useful
queries<https://bugzilla.wikimedia.org/show_bug.cgi?id=22170>and
personal information on its front page. Its table
of duplicates <https://bugzilla.wikimedia.org/duplicates.cgi> now displays bug
resolutions <https://bugzilla.wikimedia.org/show_bug.cgi?id=58749> (to
identify popular WONTFIXed requests) and
priorities<https://bugzilla.wikimedia.org/show_bug.cgi?id=56253>as
columns. The Bugzilla
etiquette <https://www.mediawiki.org/wiki/Bug_management/Bugzilla_Etiquette>was
finalized (read the
announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2014-Februa…)l>).
In Bugzilla’s taxonomy, the MobileFrontend
components<https://bugzilla.wikimedia.org/show_bug.cgi?id=61280>were
restructured and the Windows and MacOS entries in Bugzilla’s “OS”
dropdown were reordered to list recent versions first. Andre Klapper
refreshed the Annoying little
bugs<https://www.mediawiki.org/wiki/Annoying_little_bugs>page by
adding
a
section<https://www.mediawiki.org/w/index.php?title=Annoying_little_bugs…
common questions and issues of new contributors, based on Google
Code-In <https://www.mediawiki.org/wiki/Google_Code-In> experience.
*Project management tools review
<https://www.mediawiki.org/wiki/Project_management_tools/Review>*
After summarizing community input into consolidated
requirements<https://www.mediawiki.org/wiki/Project_management_tools/Rev…ts>,
Andre Klapper and Guillaume Paumier listed the different
options<https://www.mediawiki.org/wiki/Project_management_tools/Review/O…
during the consultation process. Those go from keeping the status
quo to changing a single tool, to consolidating most tools into one. They
also continued to research the main candidates by reading articles and
testing demo sites. Once the list of options has been shortened
collaboratively, the community RFC will start.
*Mentorship programs <https://www.mediawiki.org/wiki/Mentorship_programs>*
The six ongoing FOSS Outreach Program for
Women<https://www.mediawiki.org/wiki/Outreach_Program_for_Women/Round_7&…
all made good progress, and are headed to completion by the end of
the program on March 10. For more details, check their dedicated reports:
- Compacting interlanguage
links<https://www.mediawiki.org/wiki/User:Niharika/Project_Progress_Repo…
- MediaWiki Homepage
Redesign<https://www.mediawiki.org/wiki/User:Monteirobrena/MediaWiki_Hom…
- Complete the MediaWiki API development course on
Codecademy<https://www.mediawiki.org/wiki/User:Diwanshipandey/OPW_Februa…
- Clean up Parsoid round-trip testing
UI<https://www.mediawiki.org/wiki/User:5xbe/OPW_Monthly_Progress_Reports…
- Clean up tracing/debugging/logging inside
Parsoid<https://www.mediawiki.org/wiki/User:Mariapacana/OPW_Progress_Rep…
- UploadWizard: OSM
Embedding<https://www.mediawiki.org/wiki/User:Inchikutty/OPW_Internship_…
Getting Facebook Open
Academy<https://www.mediawiki.org/wiki/Facebook_Open_Academy>projects
up to speed is becoming even more complex than expected, but we
are getting there slowly. All students and mentors met at the kick-off
hackathon at Facebook headquarters on February 7−9 (see Marc-André
Pelletier’s
report<https://blog.wikimedia.org/2014/02/25/facebooks-open-academy-hack…
).
Wikimedia applied to Google Summer of Code
2014<https://www.mediawiki.org/wiki/Google_Summer_of_Code_2014>and we
were accepted. We also confirmed our participation in FOSS
Outreach Program for Women round
8<https://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_8…_8>.
We are organizing both programs simultaneously under a common umbrella, as
we did last year with great success.
*Technical communications
<https://www.mediawiki.org/wiki/Technical_communications>*
In February, Guillaume Paumier
<https://www.mediawiki.org/wiki/User:Guillom>continued to provide
ongoing communications support for the engineering
staff, and contributed to writing, simplifying, publishing and distributing
the weekly technical newsletter <https://meta.wikimedia.org/wiki/Tech/News>.
He also edited essays from Google
Code-in<https://www.mediawiki.org/wiki/Google_Code-in>students for
publication on the Wikimedia blog.
*Volunteer coordination and outreach
<https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach>*
Wikimedia completed its more ambitious participation in
FOSDEM<https://www.mediawiki.org/wiki/Events/FOSDEM>(Brussels) with
mild success. The Wikis devroom (co-organized with the
XWiki and Tiki projects), the Wikimedia stand, and *The Wikipedia
Stack*main track session achieved their basic goals in terms of
participation and
quality, but at the same time we got many ideas to do better next year.
There was more progress on the tech community
metrics<https://www.mediawiki.org/wiki/Community_metrics>front;front, and we
now have interesting data gathered around our five key
performance indicators: Who contributes
code<http://korma.wmflabs.org/browser/who_contributes_code.html>ml>;
Gerrit review
queue<http://korma.wmflabs.org/browser/gerrit_review_queue.html>ml>;
Code contributors new and
gone<http://korma.wmflabs.org/browser/code_contrib_new_gone.html>ml>;
Bugzilla response
time<http://korma.wmflabs.org/browser/bugzilla_response_time.html>ml>,
and Top
contributors<http://korma.wmflabs.org/browser/top-contributors.html>
.
*Architecture and Requests for comment process
<https://www.mediawiki.org/wiki/Architecture_process>*
We held several architecture
meetings<https://www.mediawiki.org/wiki/Architecture_meetings>to
review Requests for Comment on IRC, and continued discussion and
implementation of work begun at the architecture
summit<https://www.mediawiki.org/wiki/Architecture_Summit_2014>in
January. We also worked on improvements to the architecture guidelines
and on a draft of performance guidelines for developers.
Analytics <https://www.mediawiki.org/wiki/Analytics>
*Kraken <https://www.mediawiki.org/wiki/Analytics/Kraken>*
We continue to make progress on the Hadoop/Kafka roll-out. We’ve
encountered some issues with cross-data center latencies with Varnish-Kafka
that we are currently debugging. We are also testing the Kafka-tee
component that provides backwards compatibility for udp2log subscribers.
Finally, we are finishing a report for the Mobile team on browser
breakdowns using Kafka-provided data on Hadoop.
*Limn <https://www.mediawiki.org/wiki/Analytics/Limn>*
We’ve rolled out some minor changes that make creating dashboards easier
and more intuitive.
*Wikimetrics <https://www.mediawiki.org/wiki/Analytics/Wikimetrics>*
Work progresses on enhancing Wikimetrics into a more flexible general tool.
This month we completed work on a Vagrant deployment environment which will
make it easier for the community to work on Wikimetrics. We’ve also made
progress on the scheduler, reporting enhancements and a deployment issue.
*Data Quality <https://www.mediawiki.org/wiki/Analytics/Data_Quality>*
We’ve fixed the following production issues:
- Resolved on No sampled-1000 tsv file for 2014-02-06 on stat1002;
- Wikipedia Zero team investigated ~30% increase of number of lines zero
tsvs between 20140218 and 20140220 file;
- Wikipedia Zero team investigated on light drop in zero requests around
2014-02-08;
- Data for ULSFO Cache performance prepared for Ops blog post.
*Research and Data
<https://www.mediawiki.org/wiki/Analytics/Research_and_Data>*
[image: File:Wikimedia Research & Data Showcase - February
2014.webm]
<https://upload.wikimedia.org/wikipedia/commons/0/09/Wikimedia_Research_%26_Data_Showcase_-_February_2014.webm>
<https://commons.wikimedia.org/wiki/File:Wikimedia_Research_%26_Data_Showcase_-_February_2014.webm>
Video of the February 2014 Research Showcase
This month, we welcomed Leila
Zia<https://www.mediawiki.org/wiki/User:LZia_%28WMF%29>as the newest
addition to the team. Leila joins the Foundation as a
research scientist after completing a PhD in management science and
engineering at Stanford University. Her work will initially focus on
modeling editor lifecycles to better understand what affects their survival
and retention.
We hosted the first public Research and Data
showcase<https://www.mediawiki.org/wiki/Analytics/Research_and_Data/Show…se>,
a monthly showcase of research conducted by the team and other researchers
in the organization. This month, we presented two studies on Wikipedia
article creation
trends<https://commons.wikimedia.org/wiki/File:Wikipedia_article_creatio…
on the measurement of mobile
browsing
sessions<https://commons.wikimedia.org/wiki/File:Mobile_sessions_present…df>.
The showcase is hosted at the Wikimedia Foundation and live streamed on
YouTube every 3rd Wednesday of the month at 11.30am Pacific Time.
We attended the *17th ACM Conference on Computer-supported cooperative work
<https://en.wikipedia.org/wiki/Computer-supported_cooperative_work> and
Social Computing* (CSCW ’14) in Baltimore. Research on Wikipedia and
wiki-based collaboration has been a major focus of CSCW in the past, and
this year three Wikipedia research
papers<https://meta.wikimedia.org/wiki/Research:Newsletter/2014/February…
presented. We hosted a session to discuss collaboration opportunities
for researchers interested in tackling problems of strategic importance for
Wikimedia (a detailed CSCW ’14 report will follow on
wiki-research-l<https://lists.wikimedia.org/mailman/listinfo/wiki-resear…
).
We started creating public documentation for data sources and tools used by
the team for research and data analysis and porting docs previously hosted
on internal wikis (for example:
analytics/geolocation<https://wikitech.wikimedia.org/wiki/Analytics/Geol…
).
We continued to provide ad-hoc support to various
teams<https://trello.com/b/k5N0ivoM/research-and-data>at the
Foundation and worked closely with the Growth and Mobile teams to
prepare and review results for their respective quarterly reviews.
Kiwix <http://www.kiwix.org>
*The Kiwix project is funded and executed by Wikimedia CH
<https://meta.wikimedia.org/wiki/Wikimedia_CH>.*
For the first time, we have released a ZIM file of the entire Wikipedia in
English<http://lists.wikimedia.org/pipermail/offline-l/2014-March/001238…
all encyclopedic articles and thumbnails (download the 40GB
file via
torrent<http://download.kiwix.org/zim/wikipedia_en_all.zim.torrent>)t>).
In our announcement, we’ve also explained how we generate those archives
and advertised the tools we’ve been working with, like mwoffliner and
zimwriterfs. This month, a student also worked on the creation of ZIM files
containing TED talks. The internship is now over and was a success; ZIM
files will be published soon. Preparation work for our Usability
Hackathon<http://www.kiwix.org/wiki/Usability_Hackathon_2014>has
started.
Wikidata <https://meta.wikimedia.org/wiki/Wikidata>
*The Wikidata project is funded and executed by Wikimedia Deutschland
<https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en>.*
Wikisource now has access to the the data in Wikidata like ISBNs and the
date of birth of an author. The Lua interface for Wikidata has been
extended significantly to make it more powerful and easier to use. Support
for article badges has seen more work and is now missing mostly the user
interface part. Loading time of items on Wikidata has been improved
drastically. Everyone is asked to provide
input<https://www.wikidata.org/wiki/Wikidata:UI_redesign_input>for the
upcoming redesign of Wikidata’s user interface.
Future The engineering management team continues to update the *Deployments
<https://wikitech.wikimedia.org/wiki/Deployments>* page weekly, providing
up-to-date information on the upcoming deployments to Wikimedia sites, as
well as the *annual goals
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/2013-14_Goals>*,
listing ongoing and future Wikimedia engineering efforts.
------------------------------
*This article was written collaboratively by Wikimedia engineers and
managers. See revision history
<https://www.mediawiki.org/w/index.php?title=Wikimedia_engineering_report/2014/February&action=history>
and associated status pages. A wiki version
<https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/February>
is also available.*
--
Guillaume Paumier
Technical Communications Manager — Wikimedia Foundation
https://donate.wikimedia.org