On Wed, Oct 7, 2020 at 11:54 AM Roy Smith <roy(a)panix.com> wrote: Use a `webservice python3.7 shell` session as your dev/test environment and you will a) get the same python version as the "production" container, and b) move your dev/test workload off of the limited resources of the bastion server and onto the more scalable Kubernetes cluster. The bastions in Toolforge need to be compatible with the Grid Engine cluster because they act as job submission hosts for the grid. Today, the grid engine cluster is running Debian Stretch as its base operating system [0]. Debian Stretch is a supported release through June 2022. There is currently no scheduled work to rebuild and replace the Debian Stretch instances in Toolforge, but rest assured that this work will happen before the end of life of Debian Stretch. I expect that the Toolforge admin team will start discussing the work needed to rebuild the Toolforge bastions and grid engine instances sometime after the Debian project releases their next stable version, Bullseye [1]. [0]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Trusty_deprecation [1]: https://www.debian.org/releases/bullseye/ Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

On Wed, Oct 7, 2020 at 4:16 PM Roy Smith &lt;roy(a)panix.com&gt; wrote: This error indicates that you are running out of process quota on the bastion server due to the number of processes that you are attempting to run in parallel. We have fairly strict limits placed on each user to help prevent overwhelming the bastions. The quota is typically enough to run 2-3 small applications. I can see that you are currently logged into the dev.toolforge.org bastion (tools-sgebastion-08.tools.eqiad.wmflabs) where you have a tmux session running with four bash shells and a python process that appears to be some sort of TCP log event sync. This last process is almost certainly pushing your thread quota to the limit. This may actually be the root of some of your frustrations. Toolforge is not meant to be a replacement for a development environment on your local laptop or other server. It is intended primarily to be a runtime environment for tools. Attempting to use the bastions as your primary development environment is going to be a struggle against the various quotas and limits that have been placed on the bastions in an attempt to keep them available and responsive for people who need to start/stop/restart their tools and check log output. You might be interested in things like <https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Connecting_to_the_database_replicas_from_your_own_computer> to help you find a way to develop and test your software outside of Toolforge. Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

Bryan, Let me take a step backwards. I accept that documentation may be out of date, wrong, misleading, etc. Maybe I've been barking up the wrong tree because the out-of-date documentation led me astray. These things happen as systems evolve. So, what I want to do is develop and run web servers, written in Python/django, which can access the database mirrors (the same databases that Quarry runs against) and also make API calls. I need my users to be able to authenticate to my sever using their en.wikipedia.org <http://en.wikipedia.org/> credentials (i.e. via OAuth). I'd really like that my development environment be as similar to the production environment as possible. The more differences there are, the more complicated things become. In other words, keeping the development environment on my laptop would be sub-optimal. If you were setting out to do that, how would you set up your development and production environments, given the current state of Toolforge?