Re: GIT is down...

[Michael Tremer <michael.tremer@ipfire.org>]

Hello everyone,

I have added a ton of caching to this over the last months. It really helps, but it cannot produce any wonders.

git.ipfire.org <http://git.ipfire.org/> is pushing out over 2 TiB of data every month. Doesn’t sound a lot, but for so tiny HTML pages, that is a lot of requests. There is a large cache in front of the gitweb CGI scripts, but the crawlers barely come back to visit the same page again so it does not help too much with the bot requests. It does however help us really well with the regular clients because the index page, summary page of the individual repositories and even the latest commits are all coming from the cache.

If anyone has any other ideas how to improve this all, please let me know.

-Michael

> On 14 Sep 2025, at 09:42, Adolf Belka <adolf.belka@ipfire.org> wrote:
> 
> Hi Matthias,
> 
> On 14/09/2025 10:28, Matthias Fischer wrote:
>> Good morning,
>> sorry to say, GIT (git.ipfire.org) is down.
>> ***SNIP***
>> Oops, Something Went Wrong
>> 503 ‐ Service Unavailable
>> ***SNAP***
> 
> No it is up nut it is being heavily crawled by bots again and this is basically causing a DoS.
> 
> You just have to wait for the crawling to ease off. It is frustrating though but not so easy to block. That is the internet for you these days.
> 
> Regards,
> 
> Adolf.
> 
>> Best
>> Matthias
> 
>