[Labs-l] webservice

Mon Jul 14 12:07:12 UTC 2014

 

sure: 

the monit site is at http://mmonit.com/monit/ and docs are at
http://mmonit.com/monit/documentation/monit.html 

On 2014-07-14 15:00, labs-l-request at lists.wikimedia.org wrote: 

> Send Labs-l mailing list submissions to
> labs-l at lists.wikimedia.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.wikimedia.org/mailman/listinfo/labs-l [1]
> or, via email, send a message with subject or body 'help' to
> labs-l-request at lists.wikimedia.org
> 
> You can reach the person managing the list at
> labs-l-owner at lists.wikimedia.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Labs-l digest..."
> 
> Today's Topics:
> 
> 1. Re: webservice (Petr Bena)
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Sun, 13 Jul 2014 20:23:13 +0200
> From: Petr Bena <benapetr at gmail.com>
> To: Wikimedia Labs <labs-l at lists.wikimedia.org>
> Cc: labs-l-request at lists.wikimedia.org
> Subject: Re: [Labs-l] webservice
> Message-ID:
> <CA+4EQ5dCk=ssKuTRz=-Pgmwi89R7PPYdA5eBBCnZuLR4b=TCWQ at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
> 
> Can you link us to some manual page for that utility? Thanks
> 
> On Sat, Jul 12, 2014 at 8:45 PM, Matanya <matanya at foss.co.il> wrote:
> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi All, The best tool to make sure your websever is running is monit. It will monitor the service and bring it back up if it died. Furthermore, it can mail you on every action taken and even have in the config file an option to declare how many tries will try before giving up. Other features are available as well. On 12 ביולי 2014 15:00:37 GMT+03:00, labs-l-request at lists.wikimedia.org wrote: Send Labs-l mailing list submissions to labs-l at lists.wikimedia.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.wikimedia.org/mailman/listinfo/labs-l [1] or, via email, send a message with subject or body 'help' to labs-l-request at lists.wikimedia.org You can reach the person managing the list at labs-l-owner at lists.wikimedia.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Labs-l digest..." Today's Topics: 1. On disk use (Marc A. Pelletier) 2. Re: Webservice (Petr Bena) 3.
Re: Webservice (Marc-André Pelletier) ---------------------------------------------------------------------- Message: 1 Date: Fri, 11 Jul 2014 12:23:53 -0400 From: "Marc A. Pelletier" <marc at uberbox.org> To: Wikimedia Labs <labs-l at lists.wikimedia.org> Subject: [Labs-l] On disk use Message-ID: <53C00F99.7010804 at uberbox.org> Content-Type: text/plain; charset=ISO-8859-1 Hey all. So, a quick reminder to every labs user: project space (/data/project) is on a networked drive. While it provides a lot of space and is conveniently accesible to all instances of a project, using it /does/ incur a performance cost. Whenever a service you are running on a labs instance needs to have /local/ storage (that is, does not need to share the data with other instances), it is generally preferable to _not_ use /data/project for it. Being careful about when you use this filesystem means improved performance and reliability for everyone. On Tool Labs, where your tools can be moved arbitrarily from one node
to another, this mostly does not apply - you should be storing any persistent data in your tools' homes (which are on /data/project). It *is* possible to store data locally to the instance where the tool is running (for temporary data that does not need to persist from one run to another), provided you are careful about cleaning up after yourself. In any case, if you have any question about your disk usage or ways in which you can improve performance, don't hesitate to ask on-list or communicate with me by email or on IRC. -- Marc ------------------------------ Message: 2 Date: Fri, 11 Jul 2014 20:30:25 +0200 From: Petr Bena <benapetr at gmail.com> To: Wikimedia Labs <labs-l at lists.wikimedia.org> Subject: Re: [Labs-l] Webservice Message-ID: <CA+4EQ5ftpWksQmRF7N8iX03zOh4pAzGcoBx_y0Qhhmig5pet5w at mail.gmail.com> Content-Type: text/plain; charset=UTF-8 nope, just once in minute, crontab doesn't handle seconds, it wouldn't fire up anything but the check if it's running On Fri, Jul 11, 2014 at
12:26 AM, Hasteur Wikipedia <hasteur.wikipedia at gmail.com> wrote: Um... That's a very very bad idea. A crontab entry like that will fire multiple times a minute. What's the largest downtime that the service can tolerate? Sent from my iPhone On Jul 10, 2014, at 4:52 PM, Petr Bena <benapetr at gmail.com> wrote: what about appending this to crontab: * * * * * webservice start On Thu, Jul 10, 2014 at 5:34 PM, Tim Landscheidt
 <tim at tim-landscheidt.de> wrote: 

> Magnus Manske <magnusmanske at googlemail.com> wrote: I've been manually restarting about a dozen webservices for my
 tools in the 

> last 24h. And before you say it, some of those were Hedonil's hand-rolled
 webservice. 

> Could we PLEASE either have a Labs-official, auto- and
 self-restarting 

> webservice, or something a little more stable than lighttpd (or a
 more 

> stable way to run it)? I looked at all the tools you are a developer of and I as- sume you speak about wikidata-todo. This has some logs that appear to have indications of OOM shutdowns. You use a custom lighttpd configuration, and I'm not sure if the decision to have two PHP FCGIs doubles the memory re- quirements, at the moment using 6 GBytes out of 7 GBytes re- quested. What is clear however is that your PHP script: | 2014-07-10 14:11:39: (mod_fastcgi.c.2701) FastCGI-stderr: PHP
 Fatal error: Allowed memory size of 2621440000 bytes exhausted (tried
to allocate 71 bytes) in
/data/project/wikidata-todo/public_html/autolist2.php on line 201 

> uses almost 2.5 GByte of memory -- if I don't misread the documentation -- per /request/. Memory is cheap and we could just increase the requested limit, but I assume there are some PHP developers around who might want to have a poke at optimizing

<https://bitbucket.org/magnusmanske/wikidata-todo/src/master/public_html/autolist2.php
[3]>. 

> Regarding self-restarting web services, with continuous jobs we have a "while ! $JOB; do sleep 5; done" loop that ensures that the job is restarted if it aborts. This however does not work on OOMs that are the predominant cause of webser- vice shutdowns, as the grid engine will kill the loop as well :-). So we will probably have to start the webservice and then start a watchdog job with the webservice's job num- ber as its parameter that periodically checks that the web- service is still running and, in case, restarts the webser- vice. But to do that, jobs on execution nodes need to be able to submit jobs, and this is still pending (cf. https://bugzilla.wikimedia.org/54786 [2]). Tim _______________________________________________ Labs-l mailing list Labs-l at lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/labs-l [1] _______________________________________________ Labs-l mailing list Labs-l at lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/labs-l [1]
 _______________________________________________ Labs-l mailing list
Labs-l at lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l [1]
------------------------------ Message: 3 Date: Fri, 11 Jul 2014
15:12:46 -0400 From: Marc-André Pelletier <mpelletier at wikimedia.org> To:
labs-l at lists.wikimedia.org Subject: Re: [Labs-l] Webservice Message-ID:
<53C0372E.8080207 at wikimedia.org> Content-Type: text/plain; charset=UTF-8
On 07/11/2014 02:30 PM, Petr Bena wrote: 

> nope, just once in minute, crontab doesn't handle seconds, it
 wouldn't 

> fire up anything but the check if it's running
 I'm currently looking at some system by which tool maintainers may
specify automatic restart scripts that are sufficiently robust for
general use. Amongst other requirements, it will send an obligatory
email on every restart and will refuse to restart more than X times in a
window of Y (where X and Y are still undecided but likely to be 3 and
24h). This is to prevent problematic tools from hammering on resources
unattended. -- Marc ------------------------------
_______________________________________________ Labs-l mailing list
Labs-l at lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l [1] End of Labs-l
Digest, Vol 31, Issue 11 **************************************
-----BEGIN PGP SIGNATURE----- Version: APG v1.1.1
iQJABAEBCgAqBQJTwYI6IxxNYXRhbnlhIE1vc2VzIDxtYXRhbnlhQGZvc3MuY28u
aWw+AAoJEKzSGXfsOI0veOEP/jXtutHDwYHzIzU/zQ6tHV89xufuD0+2pKVPzqmh
VyGclpkelV/JVjHvygnovJcluqfPA/smUTBn7YgwBtaT2ElEUoeCpim/ljOdxqLE
dAAgEt9JtoBytqJxZ5z6uQkoMK1k92xjUP8U9wp9ZYqrn7i89MkuxmdaUhXp0KlM
DmoqC+Cg1XxBk6Zq7wOQYLv3Lr5uSvUynvd3rCQI0wPlsWMr+B0r5nGV1zb+DWKZ
qzFYvUW7FONdxglde3vgrMhxcl2zWEtcHz0uh/ucMcSPcoUbiUi2Oy6tPZOwscEp
nCp9Yq0nBvD//GsA/hnKXp0sLK1VtFIv0cpufevZhdoV8/4+cy36bMToRx3KBiJg
WGxfjq85vEgC9yl2+2D9Pyuz2mK+bUrKcbVyQm0FVlOyIQAKQvJ4pqff8VzlLFsR
nYLRdE+RwjsG8k4hYDVqlln0KxlNT3ZbNErNJ84ndbEXVlWreM0tTWREcof3nn/x
22ijokyz8F4AifUAo7AyYk7elJeSQVdizxptQSj0CtQXLa34R/wgwgHgxavhsPiH
Bwma4RHnVoVSZhpd1kgSOMy2lPTRK/Ww+FfwvJyZ9m18pGg0W7cm81JHDpC5BmEM
58HXmhxvcCvq1snpmcdNrac30FzotYscryC/Fed6dP2lOLiXZsGGxTdKvuAb95Qa zPlH
=YX2r -----END PGP SIGNATURE-----
_______________________________________________ Labs-l mailing list
Labs-l at lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l [1] 

------------------------------

_______________________________________________
Labs-l mailing list
Labs-l at lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l [1]

End of Labs-l Digest, Vol 31, Issue 13
**************************************

 

Links:
------
[1] https://lists.wikimedia.org/mailman/listinfo/labs-l
[2] https://bugzilla.wikimedia.org/54786
[3]
https://bitbucket.org/magnusmanske/wikidata-todo/src/master/public_html/autolist2.php
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/labs-l/attachments/20140714/4853288c/attachment.html>