[Labs-l] Recent performance issues and fixes

Tue Jan 22 07:52:59 UTC 2013

On Mon, Jan 21, 2013 at 8:16 PM, Ryan Lane <rlane at wikimedia.org> wrote:

> We've had a number of LDAP related performance issues lately. While
> reviewing the logs I noticed a very large number of queries for groups with
> relatively high gids from every instance. I also noticed than when running
> commands like: "id laner" the lookup of groups would stall shortly halfway
> through my group listing (I have about 80 groups).
>
> This can often be a sign that the search limit is being reached. We had
> increased the search limits a while back, but I've noticed recently that
> we've started reaching them again. It's a bad idea to continue to raise the
> search limits.
>
>
Just to follow up a little, it can also be a sign that the nscd cache is
too small, as well. suggested-size for groups and passwd in nscd was 211,
I've increased this to 3001, which should improve the cache hit ratio.

Unfortunately, it's not possible to get a good reading of the cache hit
ratio in nscd since many lookups don't hit the daemon, but read the cache
databases directly (we're using the shared option, which speeds things up).
That said, the maximum number of cached items for group and passwd in
nscd's statistics were way higher than the suggested-size setting.

I also noticed that I hadn't set explicit basedns for users and groups, so
I've set those to more specific OUs, which should make the searches more
efficient.

- Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/labs-l/attachments/20130121/6a32fb06/attachment-0001.html>