Scheduled downtime for Hadoop - Monday Jan 15th - 10:00 until 12:00 UTC
Hello,
We need to perform some maintenance on our primary Hadoop cluster, which will require a period of downtime. This work is scheduled for Monday Jan 15th - 10:00 until 12:00 UTC - which is a US holiday for WMF and also Wikipedia Day.
This 2 hour maintenance window has been chosen in the hope of minimising disruption for you, whilst the cluster and the various tools that depend upon it, such as Superset and JupyterLab, are largely unavailable.
The work being undertaken is a replacement
of the Hadoop nameserver hosts which, unfortunately,
requires a full cluster restart. We will be disabling ingestion
to HDFS, pausing Airflow DAGs on all instances, and stopping
production data processing pipelines, prior to the work, then
re-enabling them all afterwards. We are not expecting any gaps
in data, once the pipelines have caught up again.
If you have any queries or concerns about this work, or the
time or date is particularly in convenient for you, please don't
hesitate to let us know, so that we can look to reschedule.
Kind regards,
Ben
 |
Ben Tullis (he/him) Senior Site Reliability Engineer Wikimedia Foundation |