[Labs-admin] Maintenance during switchover time may affect labs and analytics users due to replicatin lag

Wed Apr 5 17:03:37 UTC 2017

Hello admins from Cloud and Analytics groups,

There is some extended maintenance scheduled for databases on eqiad while
it is a passive datacenter for mediawiki in the upcoming datacenter
switchover (as a reminder, the switchover will happen between 19 April and
3 May.).

Services themselves will not be affected (labs and analytics servers are
not to be touched), but due to high priority schema changes and other
cleanup work that cannot be done otherwise, replication from production
(codfw, Texas-mediawiki) to labs and analytics replicas may lag for some
hours at times. This is unavoidable because both LabsDBs and analytics is
eqiad-(Virginia datacenter) only. I hope you believe me that I am sorry for
the inconveniences, but the improvements not only will make mediawiki
better, also will impact positively your replica services (faster and more
reliable propagation of changes).

* Services affected: extended lag compared to production on labsdb replicas
(old and new) for tools and cloud, analytics store, analytics slave
(mediawiki data)
* Services definitely not affected: toolsdb, eventlogging, postgres, osm,
any other service not related to databases

Details:
* https://phabricator.wikimedia.org/T155099 (DBA task for maintenance
coordination)
* Internal draft for the blog post soon to be published:
https://office.wikimedia.org/wiki/Community_Liaisons/codfw#Initial_blog_post

Up to you if and when you want to announce it in advance. I am available
for questions an clarifications. Again, my apologies,

-- 
Jaime Crespo
<http://wikimedia.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-admin/attachments/20170405/d5d20dc5/attachment.html>