[Labs-l] Fwd: Scaling Gerrit/Git to large repos, (was Re: Gerrit service hang problem)

Thu Apr 5 21:23:47 UTC 2012

This might be worth pushing on to get applied universally
upstream. Core is huge, and it's only going to get bigger.

(I'm really trying to avoid us maintaining our own internal
fork for this kind of stuff)

-Chad

---------- Forwarded message ----------
From: Martin Fick <mfick at codeaurora.org>
Date: Thu, Apr 5, 2012 at 5:07 PM
Subject: Scaling Gerrit/Git to large repos, (was Re: Gerrit service
hang problem)
To: repo-discuss at googlegroups.com

On Wednesday, April 04, 2012 08:02:41 pm seonguk.baek wrote:
> Thanks for your reply :-)
>
> Since I added three slave servers, I got a little
> improvement. So I am thinking to add more servers for
> better performance.

You also may consider applying this patch to jgit:

https://git.eclipse.org/r/#/c/5491/1/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DateRevQueue.java

We have a similar patch running internally and it
drastically reduces the load which our larger repositories
put on our gerrit server.  If you have any repositories with
a large amount of changes and tags in them, I suspect that
anything over 50K would be a lot, we have 100K and it was
really bad until we applied this patch.  With less load,
your user requests will end quicker and your total memory
load should be reduced.

If any of your repos are that large, I would also consider
applying these git patches by René Scharfe:

http://marc.info/?l=git&m=133323194014727&w=2
http://marc.info/?l=git&m=133323194314728&w=2
http://marc.info/?l=git&m=133323763515996&w=2

to the git running on your mirrors.  Without them, you will
also be holding memory resources for much longer than you
need to on your main Gerrit server since the git receiving
end will take much longer to process any Gerrit pushes.

These might help reduce your hardware needs somewhat, they
have done miracles for us,

-Martin

--
Employee of Qualcomm Innovation Center, Inc. which is a
member of Code Aurora Forum

--
To unsubscribe, email repo-discuss+unsubscribe at googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en