@Hashar: Just wondering, are there any plans to change CI configs after moving development/code-review from gerrit to gitlab [1]? Maybe some file living in the repo directly rather than using a centralized repo like `integration/config`?
Hello,
In short it is complicated. Essentially, YES, GitLab provides a way to define your CI jobs directly in the code repository (see Gitlab CI quick start).
In longer form it is even more complicated. Given I like wall of
texts, below is some historical context regarding migrating
MediaWiki CI. It is rather long form and most can probably stop
reading at this point.
Current CI
The test infrastructure for MediaWiki is build with:
- Zuul a workflow system (X happens in Gerrit, do A, B, C tasks, report to Gerrit)
- Jenkins freestyle jobs, which you can really think as a library
of shell scripts running shell scripts serially. The jobs are
mostly invoking commands inside containers (docker
run <some args>)
- Docker images to provide the execution environment (such as the
same php version and php extensions Wikimedia uses in production)
The Zuul version we use is from 2017 and has all the CI configuration centralized (the infamous integration/config repository) though in practice the job/container delegates a lot of the logic to CI entry points such as composer test. Zuul got frozen just before a large rework by upstream. It is around that time we:
- moved from a shared build environment to Docker images
- I wrote Quibble to simplify the execution stack (instead of a hundred of scripts scattered in three different repositories)
A digression about GitLab
My grand idea at the time was to be able to trigger a Kubernetes
job with a list of parameters (provided by Zuul and consumed by
Quibble) and an execution environment (a Docker image). Then later
on find out a way to integrate with the our tool to create
container image on the fly: the novel Blubber.
The upgrade to Zuul, phase out of Jenkins and moving the execution
from WMCS to a Kubernetes cluster, those did not happen. The
foundation had other technical priorities and to be fair we did
not have the human resources to achieve any of it.
A bit later we formed a working group for the future of CI, you can see the requirements at https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/CI_Futures_WG/Requirements , and a feedback request on wikitech-l. Different solutions got evaluated and I have required Zuul version 3 to be included in the evaluation at least for comparison. Defining the CI workflow in the code repository is referred to as "self serve CI". That became popular with Travis CI. Zuul and Jenkins projects implemented similar feature around 2017 as well (Zuul version 3 and Jenkins with pipeline as code).
Proof of concepts for 3 different CI systems were build in 2019 one got picked (Argo) and we had a nice proposal: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Seakeeper_proposal . It is a good read as to what we envisioned to push.
Due to internal tensions at the foundation (others wanted to
adopt GitLab, some really hate Gerrit, others would rather
use GitHub), our Seakeeper proposal got shot at some point late in
2019. I was not in any of this discussions since I had other
matters to deal with and went briefly out of job due to some Kafkaesque
legal situation. Eventually Covid has hit early in 2020. In short
I don't think I had the mental state to be involved or consulted
in any of those discussions (nor do I think anyone would have
thought of reaching out to the de facto internal CI
expert, but I am exaggerating and ranting at this point. I
apologize).
GitLab entered the foundation annual plan for July 2020 - June 2021. You can read the public announce at https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/S6GHXFS2GD2XFCDXT47KDY26RFHCK7R3/ . There were some backslashes about lack of a public consultation (and rightfully so) but in the end the decision had already been made. To my knowledge all people who were actively pushing for GitLab have since left the foundation or moved to different departments.
Self serve CI!
All of that to say that, in GitLab, one can define the CI
workflow directly from the repository. I think the documentation
is https://docs.gitlab.com/ee/ci/pipelines/merge_request_pipelines.html
and that system is entirely different than our current
Zuul/Jenkins/Container images. We can surely reuse the existing
images with minor adaptations, but everything else has to be
migrated and most probably rewritten from scratch (unless we get
smart enough to convert the Zuul / Jenkins jobs yaml definition to
Gitlab CI pipelines).
How are we going to migrate MediaWiki to GitLab?
Well that is a good question, and in all transparency nobody
knows. During the GitLab consultation I have warned the GitLab
community edition we envisioned is missing critical features (
some at https://www.mediawiki.org/wiki/Topic:Vv0c2rqyzfy1dos2
) and internally I and others very boldly warned about the huge
cost of migration with either or both of:
- a radical change of how we develop MediaWiki (lack of gating changes, no cross repositories dependencies, lack of search)
- a lot of tooling on top of GitLab
For MediaWiki those discussions haven't started yet.
Hopeful conclusion
The conclusion is that moving the CI definitions toward the code
repositories is definitely a feature we seek. GitLab projects
already have self serve CI. Before GitLab we had achieved it for
MediaWiki services with PipelineLib/Blubber
and there is work going on right now to add it to GitLab. Once
completed, that would open the migration of MediaWiki services to
GitLab. For MediaWiki the future remains to be written though, one
way or the other it would be easier for developers to define the
tests workflows.
cheers,
Antoine "hashar" Musso