[Cloud] Migrating Tools Off Grid Engine And The Way Forward

Hello! Earlier this year, WMCS initiated the process to migrate tools off the grid[0]. We also published a series of blog posts explaining further the reasoning behind this action[1] We encouraged maintainers to move to Kubernetes if they could but also made available Debian Buster GridEngine for those tools who were blocked or otherwise unable to migrate to Kubernetes at that time. We are aware that not all workloads can easily move from the grid to Kubernetes.[2] For some of the current grid workflows, there may be no 1:1 functionality match on Kubernetes. Work is underway to address most of these issues[3] We’re putting together a use case continuity table showing GridEngine workloads and their equivalent Kubernetes workloads[4] [image: case continuity.PNG] To help track the specific migration work, we created a Phabricator ticket(project tag: grid-engine-to-k8s-migration[5]) for each tool that is currently running on GridEngine. With a ticket for each tool on GridEngine, we hope to collect specific blocking issues and have the team work on addressing them. We encourage maintainers to reach out if you need help or find you are blocked by missing features. We noticed, after receiving notifications for these tickets, some of you wondered whether the grid is being shut down immediately. This is not the case. We will work with tool maintainers to ensure all tools safely move off the grid(or are safely shutdown), only then will we start looking at decommissioning the grid. Apologies to those who felt spammed by the ticket creation process and got worried about the future of their projects. We should have communicated better around this process. === Way Forward === The working draft for GridEngine plans and timeline can be found here[6] If you need further clarifications, reach out to us on the ticket for your specific tool on Phabricator or reach out via any of our communication channels[7] Thanks! ---------- [0]: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/… [1]: https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/ [2]: https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/Enhanceme… GridEngine_plans_and_timeline#Use_case_continuity [3]: https://phabricator.wikimedia.org/T194332 [4]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation#… [5]: https://phabricator.wikimedia.org/project/profile/6135/ [6]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation [7]: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/About_Toolforge#Commun… -- Seyram Komla Sapaty Developer Advocate Wikimedia Cloud Services