Re: ipblacklist V2

[Tim FitzGeorge <ipfr@tfitzgeorge.me.uk>]

[-- Attachment #1: Type: text/plain, Size: 7373 bytes --]

Hello,

On 10/02/2022 16:48, Michael Tremer wrote:
> Hello Rob,
> 
>> On 10 Feb 2022, at 15:12, Rob Brewer <ipfire-devel(a)grantura.co.uk> wrote:
>>
>> Hi Michael
>>
>> On Thursday 10 February 2022 09:41 Michael Tremer wrote:
>>
>>> Hello Rob,
>>>
>>>
>>> Good to hear that you are in touch. I would like to invite Tim to join the
>>> conversation on here. I am sure he has a couple of thoughts to contribute
>>> and I hope he can find the time.
>>>
>>>
>>> I assume you are talking about this here?
>>>
>>>
>> https://git.ipfire.org/?p=people/timf/ipfire-2.x.git;a=shortlog;h=refs/heads/ipblacklist
>>
>> Yes Tim's link to his ipfire git pages which links your URL.
> 
> Okay, I just set up your very own repository over here:
> 
>   https://git.ipfire.org/?p=people/helix/ipfire-2.x.git;a=summary
> 
> I will send you a private email with your account stuff.
> 
>>>
>>> That would have been one of my first questions having looked at my emails
>>> again: Get the code into some Git repository.
>>>
>>> This is a large patchset and it is very difficult to scroll up and down to
>>> review it. Uploading it to a Git repository that is browsable in a web
>>> browser somewhere would be a lot better and we can put any patches on top
>>> of the branch, so that we only will have smaller changes to review and not
>>> a whole patchset again and again.
>>>
>> I think the very large patch which gave you problems in May 2020 and linked 
>> to below and (PATCH v2 0/8 on Patchwork), patched the abandoned version 1 of 
>> ipblacklist to version 2. The resulting V2 patches:
>>
>> ipblacklist: Build infrastructure
>> ipblacklist: Modifications to system
>> ipblacklist: Ancillary files
>> ipblacklist: WUI menus, language file etc
>> ipblacklist: WUI Log details page
>> ipblacklist: WUI Log page
>> ipblacklist: WUI Settings page
>> ipblacklist: Main script
>>
>> are on:
>> https://git.ipfire.org/?p=people/timf/ipfire-2.x.git;a=shortlog;h=refs/heads/ipblacklist
>>
>> I think these were (at May 2020) pretty much ready to go and would only need 
>> updating if the core files have changed in the meantime. To date the only 
>> required patch I am aware of would be to ipblacklists.dat to reflect the 
>> changes in the latest version of IPFire. (There could be others as I only 
>> patched  my system files rather than replaced them)
> 
> What changes are those?
> 
>>> Do you have a Git repository somewhere? Would you like me to set up your
>>> IPFire account so that you can use our servers?
>>>
>>
>> I have a Git Development Environment on this computer (Debian Sid 5.16.0-1-
>> amd64 16GB memory) currently at core163 but is a bit short of room as the 
>> disk is showing 95% full. Your offer of an account on IPFire might be a 
>> sensible alternative. 
> 
> See above.
> 
>>> Do you have experience with Git?
>>>
>>> We would need to rebase the branch onto next (which Adolf has already
>>> pointed out), but I don’t think this would be a problem because we are
>>> mainly adding new code and don’t modify too much existing stuff here.
>>>
>> I don't think that would be a problem.
> 
> Great!
> 
>>
>>>> You may be interested in one of the modification I have made to
>>>> ipblacklist, is to add an additional local blacklist to the sources file
>>>> to get a
>>>> blocklist from a web server on my local network.  This is populated by a
>>>> script which greps the mail server logs for SMTP Auth attacks and has
>>>> been particularly useful in protecting the mail server from a recent
>>>> botnet attack where the offending ip addresses have been recycled every
>>>> one to three weeks. Currently the blocklist contains about 3000 ip
>>>> addresses and has blocked nearly 2000 smtp auth attempts so far to-day.
>>>>
>>>> I also use fail2ban and Banish to manage iptables blocks on the firewall.
>>>
>>> This is kind of a fail2ban but on the firewall. Since this patchset is
>>> already so large and there has been a custom blocklist existing before
>>> which got removed, I would push this project back a little bit until we
>>> have a base that we can add new features to.
>>>
>> I wasn't suggesting that we add this to the current ipblacklist version. 
>> However it only requires 1 extra resource added to the sources file so the 
>> modifications are minimal but would have limited user appeal and it does 
>> require some local processing of server lo files. However for my botnet 
>> attack it is being very effective.
> 
> Okay. It is great to see that this works wonders already :)
> 
>>> I am not entirely convinced that this functionality scales well across all
>>> users. How would they move the logs to the firewall? We don’t have a
>>> simple API, but if we did, we would not have a system to authenticated
>>> other servers. This would be a project that I would find a little bit more
>>> complicated and we would need a couple more pieces in the puzzle before we
>>> are ready for it.
>>>
>> I fully agree.

This sounds as if it does the same sort of thing as something I had in my first patchset.  I added an extra rule to the input policy chain that added the address to an ipset if the number of dropped packets exceeded a threshold.  This runs completely within iptables/ipset.

iptables( "-I ${autoblacklist}_BLOCK -m set --match-set $autoblacklist src -j SET --add-set $autoblacklist src --exist" );
iptables( "-I ${autoblacklist}_BLOCK -m set --match-set $autoblacklist dst -j SET --add-set $autoblacklist dst --exist" );
iptables( "-I POLICYIN 1 -i $red_iface -m hashlimit --hashlimit-mode srcip --hashlimit-above $settings{BLOCK_THRESHOLD}/hour --hashlimit-name $autoblacklist -j SET --add-set $autoblacklist src" );

>>
>>>> The last communication I could find between yourself and Tim was in May
>>>> 2020. https://lists.ipfire.org/pipermail/development/2020-May/007822.html
>>>
>>> Thanks for finding this. Indeed the conversation just ended in nothingness
>>> due to lack of time of everybody I would suspect.
>>>
>>> I could not find anything on the list that I would consider a blocker.
>>> There are however some smaller things like translations and probably there
>>> will be a couple of minor bugs and some improvements to the look and feel.
>>>
>> There are a couple of points we need to consider:
>>
>> 1) IPBlacklist does not work very well if Tim's  ipfblocklist add-on is also 
>> installed. My view is that the add-on should be removed before IPBlacklist 
>> can be applied. Can the add-on be automatically removed on installaion and 
>> should we transfer the settings info from ipfbocklist to ipblacklist?
> 
> Yes, in theory we could remove any old files in the updater and install our own ones.
> 
>> 2) I added a init script to my firewall which doesn't seem to be present on 
>> Tim's patches. I'm not sure if this is needed as it will be started by fcron 
>> or changes made in the WUI but won't be instantly available on re-boot. Do 
>> you have any thoughts on this?
> 

I don't think this is needed - the change to the firewall init script should call the ipblacklist script at the correct time.

> What does the script do?
> 
>>> So, can we start with rebasing the Git branch, please?
>>>
>>
>> No problem.
>>
>> Rob
>>
> 
> -Michael

Tim