[Cloud-admin] Re: What if we stop doing Openstack

Hi there, Last year, we starting evaluating how we could refresh the way we relate (deploy, maintain, upgrade) our Openstack deployment for Cloud VPS [0]. One of the most compelling options we found was to run Openstack inside Kubernetes, using an upstream project called openstack-helm. But... What if we stopped doing Openstack at all? To clarify, the base idea I had is: * deploy Kubernetes to a bunch of hosts in one of our Wikimedia datacenters ** we know how to do it! ** this would be the base, undercloud, or bedrock, whatever. * deploy ceph next to k8s (maybe, inside even?) ** ceph would remain the preferred network storage solution * deploy some kind of k8s multiplexing tech ** example: https://www.vcluster.com/ but there could be others ** using this create a dedicated k8s cluster for each project, for example: toolforge/toolsbeta/etc * Inside this new VM-less toolforge, we can retain pretty much the same functionalities as today: ** a container listening on 22/tcp with kubectl & toolforge cli installed can be the login bastion ** NFS server can be run on a container, using ceph ** toolsDB can be run on a container. Can't it? Or maybe replace it with other k8s-native solution * If we need any of the native openstack components, for example Keystone or Swift we may run them on an standalone fashion inside k8s. * We already have some base infrastructure (and knowledge) that would support this model. We have cloudlbs, cloudgw, we know how to do ceph, etc. * And finally, and most important: the community. The main question could be: ** Is there any software running on Cloud VPS virtual machines that cannot run on a container in kubernetes? I wanted to start this email hoping that I would collect a list of use cases, blockers, and strong opinions about why running Openstack is important (or not). I'm pretty sure I'm overlooking some important thing. I plan to document all this on wikitech, and/or maybe phabricator. You may ask: and why stop doing openstack? I will answer that in a different email to keep this one as short as possible. Looking forward to your counter-arguments. Thanks! regards. [0] https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/Enhanceme… _______________________________________________ Cloud-admin mailing list -- cloud-admin(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-admin.lists.wikimedia.org/