Re: ipblacklist V2

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 7707 bytes --]

Hello Tim,

Welcome back :)

> On 12 Feb 2022, at 21:29, Tim FitzGeorge <ipfr(a)tfitzgeorge.me.uk> wrote:
> 
> Hello,
> 
> On 10/02/2022 16:48, Michael Tremer wrote:
>> Hello Rob,
>> 
>>> On 10 Feb 2022, at 15:12, Rob Brewer <ipfire-devel(a)grantura.co.uk> wrote:
>>> 
>>> Hi Michael
>>> 
>>> On Thursday 10 February 2022 09:41 Michael Tremer wrote:
>>> 
>>>> Hello Rob,
>>>> 
>>>> 
>>>> Good to hear that you are in touch. I would like to invite Tim to join the
>>>> conversation on here. I am sure he has a couple of thoughts to contribute
>>>> and I hope he can find the time.
>>>> 
>>>> 
>>>> I assume you are talking about this here?
>>>> 
>>>> 
>>> https://git.ipfire.org/?p=people/timf/ipfire-2.x.git;a=shortlog;h=refs/heads/ipblacklist
>>> 
>>> Yes Tim's link to his ipfire git pages which links your URL.
>> 
>> Okay, I just set up your very own repository over here:
>> 
>>  https://git.ipfire.org/?p=people/helix/ipfire-2.x.git;a=summary
>> 
>> I will send you a private email with your account stuff.
>> 
>>>> 
>>>> That would have been one of my first questions having looked at my emails
>>>> again: Get the code into some Git repository.
>>>> 
>>>> This is a large patchset and it is very difficult to scroll up and down to
>>>> review it. Uploading it to a Git repository that is browsable in a web
>>>> browser somewhere would be a lot better and we can put any patches on top
>>>> of the branch, so that we only will have smaller changes to review and not
>>>> a whole patchset again and again.
>>>> 
>>> I think the very large patch which gave you problems in May 2020 and linked 
>>> to below and (PATCH v2 0/8 on Patchwork), patched the abandoned version 1 of 
>>> ipblacklist to version 2. The resulting V2 patches:
>>> 
>>> ipblacklist: Build infrastructure
>>> ipblacklist: Modifications to system
>>> ipblacklist: Ancillary files
>>> ipblacklist: WUI menus, language file etc
>>> ipblacklist: WUI Log details page
>>> ipblacklist: WUI Log page
>>> ipblacklist: WUI Settings page
>>> ipblacklist: Main script
>>> 
>>> are on:
>>> https://git.ipfire.org/?p=people/timf/ipfire-2.x.git;a=shortlog;h=refs/heads/ipblacklist
>>> 
>>> I think these were (at May 2020) pretty much ready to go and would only need 
>>> updating if the core files have changed in the meantime. To date the only 
>>> required patch I am aware of would be to ipblacklists.dat to reflect the 
>>> changes in the latest version of IPFire. (There could be others as I only 
>>> patched  my system files rather than replaced them)
>> 
>> What changes are those?
>> 
>>>> Do you have a Git repository somewhere? Would you like me to set up your
>>>> IPFire account so that you can use our servers?
>>>> 
>>> 
>>> I have a Git Development Environment on this computer (Debian Sid 5.16.0-1-
>>> amd64 16GB memory) currently at core163 but is a bit short of room as the 
>>> disk is showing 95% full. Your offer of an account on IPFire might be a 
>>> sensible alternative. 
>> 
>> See above.
>> 
>>>> Do you have experience with Git?
>>>> 
>>>> We would need to rebase the branch onto next (which Adolf has already
>>>> pointed out), but I don’t think this would be a problem because we are
>>>> mainly adding new code and don’t modify too much existing stuff here.
>>>> 
>>> I don't think that would be a problem.
>> 
>> Great!
>> 
>>> 
>>>>> You may be interested in one of the modification I have made to
>>>>> ipblacklist, is to add an additional local blacklist to the sources file
>>>>> to get a
>>>>> blocklist from a web server on my local network.  This is populated by a
>>>>> script which greps the mail server logs for SMTP Auth attacks and has
>>>>> been particularly useful in protecting the mail server from a recent
>>>>> botnet attack where the offending ip addresses have been recycled every
>>>>> one to three weeks. Currently the blocklist contains about 3000 ip
>>>>> addresses and has blocked nearly 2000 smtp auth attempts so far to-day.
>>>>> 
>>>>> I also use fail2ban and Banish to manage iptables blocks on the firewall.
>>>> 
>>>> This is kind of a fail2ban but on the firewall. Since this patchset is
>>>> already so large and there has been a custom blocklist existing before
>>>> which got removed, I would push this project back a little bit until we
>>>> have a base that we can add new features to.
>>>> 
>>> I wasn't suggesting that we add this to the current ipblacklist version. 
>>> However it only requires 1 extra resource added to the sources file so the 
>>> modifications are minimal but would have limited user appeal and it does 
>>> require some local processing of server lo files. However for my botnet 
>>> attack it is being very effective.
>> 
>> Okay. It is great to see that this works wonders already :)
>> 
>>>> I am not entirely convinced that this functionality scales well across all
>>>> users. How would they move the logs to the firewall? We don’t have a
>>>> simple API, but if we did, we would not have a system to authenticated
>>>> other servers. This would be a project that I would find a little bit more
>>>> complicated and we would need a couple more pieces in the puzzle before we
>>>> are ready for it.
>>>> 
>>> I fully agree.
> 
> This sounds as if it does the same sort of thing as something I had in my first patchset.  I added an extra rule to the input policy chain that added the address to an ipset if the number of dropped packets exceeded a threshold.  This runs completely within iptables/ipset.
> 
> iptables( "-I ${autoblacklist}_BLOCK -m set --match-set $autoblacklist src -j SET --add-set $autoblacklist src --exist" );
> iptables( "-I ${autoblacklist}_BLOCK -m set --match-set $autoblacklist dst -j SET --add-set $autoblacklist dst --exist" );
> iptables( "-I POLICYIN 1 -i $red_iface -m hashlimit --hashlimit-mode srcip --hashlimit-above $settings{BLOCK_THRESHOLD}/hour --hashlimit-name $autoblacklist -j SET --add-set $autoblacklist src" );
> 
>>> 
>>>>> The last communication I could find between yourself and Tim was in May
>>>>> 2020. https://lists.ipfire.org/pipermail/development/2020-May/007822.html
>>>> 
>>>> Thanks for finding this. Indeed the conversation just ended in nothingness
>>>> due to lack of time of everybody I would suspect.
>>>> 
>>>> I could not find anything on the list that I would consider a blocker.
>>>> There are however some smaller things like translations and probably there
>>>> will be a couple of minor bugs and some improvements to the look and feel.
>>>> 
>>> There are a couple of points we need to consider:
>>> 
>>> 1) IPBlacklist does not work very well if Tim's  ipfblocklist add-on is also 
>>> installed. My view is that the add-on should be removed before IPBlacklist 
>>> can be applied. Can the add-on be automatically removed on installaion and 
>>> should we transfer the settings info from ipfbocklist to ipblacklist?
>> 
>> Yes, in theory we could remove any old files in the updater and install our own ones.
>> 
>>> 2) I added a init script to my firewall which doesn't seem to be present on 
>>> Tim's patches. I'm not sure if this is needed as it will be started by fcron 
>>> or changes made in the WUI but won't be instantly available on re-boot. Do 
>>> you have any thoughts on this?
>> 
> 
> I don't think this is needed - the change to the firewall init script should call the ipblacklist script at the correct time.
> 
>> What does the script do?
>> 
>>>> So, can we start with rebasing the Git branch, please?
>>>> 
>>> 
>>> No problem.
>>> 
>>> Rob
>>> 
>> 
>> -Michael
> 
> Tim