Hi
Last month a dozen of Kiwix developers have attend to our Spring 2017
hackathon in Lyon, France. All these developers represent the broad
spectrum of Kiwix/openZIM activities, from the scrapping to the mobile
apps, through core lib development.
During a whole week we all have lived and worked together to help the
whole project to make a significant step forward. Here is the essence of
what we have achieved to do:
*First Chrome extension*: this is a pure Web tech Kiwix reader. Still
limited in features but working and already published on the Chrome
Store
https://chrome.google.com/webstore/detail/kiwix/donaljnlmapmngakoipdmehbfci….
Give it a try and report any problem here:
https://github.com/kiwix/kiwix-html5/issues
*First Firefox extension*: sharing the same code base as the Chrome
extension, it is still in validation but should be publicly available
soon. Impatient users might install a "nightly" from here
http://download.kiwix.org/nightly/. Both Firefox/Chrome extensions have
been initiated by the project Evopedia which has merged with Kiwix last
year.
*Gutenberg scraper* improved: Developed 3 years ago, the original
scrapper has been polished and the (big) list of tickets has been
cleared. The code has been published in the Python repository
https://pypi.python.org/pypi/gutenberg2zim and a Docker image has been
created https://hub.docker.com/r/openzim/gutenberg/. New ZIM files are
currently building and will be published soon.
*First release of Sotoki*: Sotoki is our StackExchange scrapper, which
means that we can now propose ZIM files of all the SE web sites
https://stackexchange.com/sites. The most famous of them being
StackOverflow. The script has been published in Python
https://pypi.python.org/pypi/sotoki and a Docker image created
https://hub.docker.com/r/openzim/sotoki/. New ZIM files are currently
building and will be published soon.
*Youtube scrapper has been improved*: Our Youtube scrapper has been a
bit improved and published in Python repository and a Docker image has
been created https://hub.docker.com/r/openzim/youtube/.
*openedx scraper stub*: First step have been done, we currently search a
sponsor to work on this next Summer and release a first version at Fall.
*Phet scrapper UI improved*: The UI of the Phet ZIM file has been really
improved. You can check it with the last versions of the files
http://download.kiwix.org/zim/phet/. A docker image has been created
https://hub.docker.com/r/openzim/phet/. Phets Android app will be
updated in the next weeks
https://play.google.com/store/apps/details?id=org.kiwix.kiwixcustomphet
*Android apps*: many improvements have been brought to the apps, the
most visible concerning the continuous integration and the testing. But
other massive changes have been done to have a better MVC approach and
bookmarking system. Many of these improvements should be visible in
Kiwix for Android 2.3 to be released in June.
*Fist Kiwix Apache module* has been created as an alternative of running
kiwix-serve behind a reverse proxy. So far it does not provide as much
features as kiwix-serve but is definitely easier to configure and really
promising. Have a look to the git repository here
https://github.com/kiwix/kiwix-apache
*MWoffliner improved*: it benefits now from an option to scrape remote
Mediawiki, even if they do not have Parsoid installed. This is an
opportunity for creating many new ZIM files - and we will. MWoffliner
(as a script and a library) is now also available as a node.js package
https://www.npmjs.com/package/mwoffliner and we have created a Docker
image https://hub.docker.com/r/openzim/mwoffliner/. We have also created
a Docker image for zimwriterfs
https://hub.docker.com/r/openzim/zimwriterfs/.
*pibox-installer*: our intern, working on our future solution to allow
everyone (also non-tech) to build there own customed offline wifi
hotspot, was also part of the hackathon. He made a few progresses on his
project during that week. Follow his daily work at
https://framagit.org/ideascube/pibox-installer
*Khan Academy video scrapper almost finished*: This effort had been
started in January last year. It has been almost finished during the
hackathon. A few commits more and we will be able to generate ZIM files
for Khan academy videos. Please be a bit patient if you need the ZIM
files, here is the git repo https://github.com/openzim/kalite
*libzim improvements*: we have moved the Xapian fulltext engine
technology from the Kiwix-lib to the libzim (as an option). At this
occasion many improvements have been done regarding the search and the
whole integration between the libzim, libkiwix and the multiple ports.
*First release of zip2zim* which is a small service able to convert a
custom ZIP (with HTML/JS/CSS/pictures inside) in the corresponding ZIM
file. The code is available here https://github.com/openzim/zip2zim and
we have already created a Docker image
https://hub.docker.com/r/openzim/zip2zim/. We plan to setup an online
service soon.
Last but now least, the hackathon has concluded ~ 6 months long effort
to reorganize all our git repositories. All the code we have ever
produced and still produce is now available on GitHub in two organisations:
* openZIM for the low level ZIM related readers and scrappers
https://github.com/openzim/
* Kiwix for all the "high-level" Kiwix specific ports and solutions
https://github.com/kiwix
This was a long list, thank you for reading it to the end, but a dozen
of talented and committed developers can really to a lot of work during
a whole week. If you want to get a bit more details about the hackathon
have a look to that page: wiki.kiwix.org/wiki/Hackathon_Spring_2017
Next one will be this summer @Wikimania in Canada/US, here is the
organisation page:http://wiki.kiwix.org/wiki/Hackathon_Wikimania_2017
Regards
Emmanuel
--
Kiwix - Wikipedia Offline & more
* Web: http://www.kiwix.org
* Twitter: https://twitter.com/KiwixOffline
* more: http://www.kiwix.org/wiki/Communication