[Labs-l] NFS server network capacity upgrade

Sat May 24 07:56:12 UTC 2014

Hoi,
Nice in theory. However tools DO die when others produce too much shit for
the server to handle.

In my mind the most important thing is for Labs to be operational. Worrying
about dimes and cents is too expensive when it is at the cost of a
diminished service to Labs users.

Yes, even when performance is always ensured it pays to target bad
practices because sure as hell, some things do need improvements and it
pays to make sure that software gets optimised.
Thanks,
     GerardM

On 24 May 2014 09:39, Petr Bena <benapetr at gmail.com> wrote:

> what about doing some steps to optimize current resource usage so that
> it's not needed to put more and more money to increase the HW
> resources?
>
> for example I believe there is a number of tools that are using nfs
> servers in insane way, for example generating tons of temporary data
> that could be instead stored to /tmp instead of /data/project also the
> static binaries that are in /data/project could be probably cached in
> memory somehow so that they don't need to be loaded over network
> everytime the task restart.
>
> Perhaps installing sar-like monitoring tool on nfs server would help
> to discover which tool uses the nfs most and such a report could help
> developers of these tools to figure out where is a need for
> optimization. I myself have some idea of how labs work so my own tools
> are usually very optimized to use these network resources (and even
> disk storage) as less as possible, but others might not be aware of
> that and may need some help optimizing these.
>
> On Fri, May 23, 2014 at 6:28 PM, Marc A. Pelletier <marc at uberbox.org>
> wrote:
> > Hello everyone,
> >
> > In the following week or two, we are planning on adding another bound
> > network port to increase the NFS server's bandwidth (which is,
> > currently, saturating at regular interval).
> >
> > This will imply a short period of downtime (on the order of 10 minutes
> > or so) during which no NFS service will be provided.  In theory, this
> > will result in file access simply stalling and resuming at the end of
> > the outage, but processes that have timeouts may be disrupted (in
> > particular, web service access will likely report gateway issues during
> > that interval).
> >
> > While this is not set in stone, I am aiming for Friday, May 30 at 18:00
> > UTC for the downtime.  I will notify this list with a confirmation or a
> > new schedule in at least three days in advance.
> >
> > Thanks for your patience,
> >
> > -- Marc
> >
> > _______________________________________________
> > Labs-l mailing list
> > Labs-l at lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/labs-l
>
> _______________________________________________
> Labs-l mailing list
> Labs-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/labs-l
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/labs-l/attachments/20140524/75010a5a/attachment-0001.html>