Re: Status emails and IP Blocklists

Re: Status emails and IP Blocklists

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 1181 bytes --]

Hello Tim, hello Michael,

> 
>> The second addon handles the setting up and updating of IP Address
>> Blocklists in the firewall.  It includes options to select which lists
>> to use, and some control over how frequently to check for updates.
> 
> I guess Peter might be quite excited about this :)
I _am_ excited about this indeed. Especially the "Emerging FW" combined
list sounds very interesting. Dropping bogon traffic is also a good
idea, as it prevents some hijacked BGP allocation stuff.

> 
> I personally do not have much use for this, but again, why should this not
> become part of IPFire?
> 
@Michael: Why do you have no use for this? Speaking about the mentioned
Emerging FW list, enabling it as a default sounds reasonable to me. Networks
listed there usually are so bad one even does not want to route or peer
to it (DROP = Don't route or peer). :-)

Could we enable the bogon list as a default for dial-up interfaces in
IPFire 3.x ?

Thanks, and best regards,
Peter Müller
-- 
Microsoft DNS service terminates abnormally when it recieves a response
to a DNS query that was never made.  Fix Information: Run your DNS
service on a different platform.
		-- bugtraq

Re: Status emails and IP Blocklists

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 1855 bytes --]

Hey,

> On 1 Dec 2018, at 20:18, Peter Müller <peter.mueller(a)link38.eu> wrote:
> 
> Hello Tim, hello Michael,
> 
>> 
>>> The second addon handles the setting up and updating of IP Address
>>> Blocklists in the firewall.  It includes options to select which lists
>>> to use, and some control over how frequently to check for updates.
>> 
>> I guess Peter might be quite excited about this :)
> I _am_ excited about this indeed. Especially the "Emerging FW" combined
> list sounds very interesting. Dropping bogon traffic is also a good
> idea, as it prevents some hijacked BGP allocation stuff.
> 
>> 
>> I personally do not have much use for this, but again, why should this not
>> become part of IPFire?
>> 
> @Michael: Why do you have no use for this? Speaking about the mentioned
> Emerging FW list, enabling it as a default sounds reasonable to me. Networks
> listed there usually are so bad one even does not want to route or peer
> to it (DROP = Don't route or peer). :-)

Well, that one maybe :) I forgot that we could use this on the IPFire
Infrastructure…

I am not sure if this should be enabled by default. We deliberately do not
ship the firewall in the most secure way it is possible. Then, we would not
allow any traffic to pass whatsoever, but it makes the setup rather difficult
and you might be running into unexpected issues.

But we should strongly recommend enabling this.

> Could we enable the bogon list as a default for dial-up interfaces in
> IPFire 3.x ?

Not only dial-up, but this probably would not be a dynamic list, but
rather a substantial part of the firewall.

-Michael

> Thanks, and best regards,
> Peter Müller
> -- 
> Microsoft DNS service terminates abnormally when it recieves a response
> to a DNS query that was never made.  Fix Information: Run your DNS
> service on a different platform.
> 		-- bugtraq

Re: Status emails and IP Blocklists

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 2368 bytes --]

Hello Michael,

> Hey,
> 
>> On 1 Dec 2018, at 20:18, Peter Müller <peter.mueller(a)link38.eu> wrote:
>>
>> Hello Tim, hello Michael,
>>
>>>
>>>> The second addon handles the setting up and updating of IP Address
>>>> Blocklists in the firewall.  It includes options to select which lists
>>>> to use, and some control over how frequently to check for updates.
>>>
>>> I guess Peter might be quite excited about this :)
>> I _am_ excited about this indeed. Especially the "Emerging FW" combined
>> list sounds very interesting. Dropping bogon traffic is also a good
>> idea, as it prevents some hijacked BGP allocation stuff.
>>
>>>
>>> I personally do not have much use for this, but again, why should this not
>>> become part of IPFire?
>>>
>> @Michael: Why do you have no use for this? Speaking about the mentioned
>> Emerging FW list, enabling it as a default sounds reasonable to me. Networks
>> listed there usually are so bad one even does not want to route or peer
>> to it (DROP = Don't route or peer). :-)
> 
> Well, that one maybe :) I forgot that we could use this on the IPFire
> Infrastructure…
Spamhaus SBL also covers networks listed in DROP (return code: 127.0.0.9),
so we already have it in use there. Further, our mail server rejects messages
relayed through such an IP at some point. Needless to say, direct delivery
attempts from an IP listed anywhere at Spamhaus are rejected.

See /etc/rspamd/local.d/force_actions.conf and https://www.spamhaus.org/faq/section/DROP%20FAQ#435
for details.
> 
> I am not sure if this should be enabled by default. We deliberately do not
> ship the firewall in the most secure way it is possible. Then, we would not
> allow any traffic to pass whatsoever, but it makes the setup rather difficult
> and you might be running into unexpected issues.
> 
> But we should strongly recommend enabling this.
Okay.
> 
>> Could we enable the bogon list as a default for dial-up interfaces in
>> IPFire 3.x ?
> 
> Not only dial-up, but this probably would not be a dynamic list, but
> rather a substantial part of the firewall.
ACK.

Thanks, and best regards,
Peter Müller
-- 
Microsoft DNS service terminates abnormally when it recieves a response
to a DNS query that was never made.  Fix Information: Run your DNS
service on a different platform.
		-- bugtraq

Re: Status emails and IP Blocklists

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 2597 bytes --]

Hey,

> On 2 Dec 2018, at 12:08, Peter Müller <peter.mueller(a)link38.eu> wrote:
> 
> Hello Michael,
> 
>> Hey,
>> 
>>> On 1 Dec 2018, at 20:18, Peter Müller <peter.mueller(a)link38.eu> wrote:
>>> 
>>> Hello Tim, hello Michael,
>>> 
>>>> 
>>>>> The second addon handles the setting up and updating of IP Address
>>>>> Blocklists in the firewall.  It includes options to select which lists
>>>>> to use, and some control over how frequently to check for updates.
>>>> 
>>>> I guess Peter might be quite excited about this :)
>>> I _am_ excited about this indeed. Especially the "Emerging FW" combined
>>> list sounds very interesting. Dropping bogon traffic is also a good
>>> idea, as it prevents some hijacked BGP allocation stuff.
>>> 
>>>> 
>>>> I personally do not have much use for this, but again, why should this not
>>>> become part of IPFire?
>>>> 
>>> @Michael: Why do you have no use for this? Speaking about the mentioned
>>> Emerging FW list, enabling it as a default sounds reasonable to me. Networks
>>> listed there usually are so bad one even does not want to route or peer
>>> to it (DROP = Don't route or peer). :-)
>> 
>> Well, that one maybe :) I forgot that we could use this on the IPFire
>> Infrastructure…
> Spamhaus SBL also covers networks listed in DROP (return code: 127.0.0.9),
> so we already have it in use there. Further, our mail server rejects messages
> relayed through such an IP at some point. Needless to say, direct delivery
> attempts from an IP listed anywhere at Spamhaus are rejected.
> 
> See /etc/rspamd/local.d/force_actions.conf and https://www.spamhaus.org/faq/section/DROP%20FAQ#435
> for details.

I know, but I meant for outgoing connections...

>> 
>> I am not sure if this should be enabled by default. We deliberately do not
>> ship the firewall in the most secure way it is possible. Then, we would not
>> allow any traffic to pass whatsoever, but it makes the setup rather difficult
>> and you might be running into unexpected issues.
>> 
>> But we should strongly recommend enabling this.
> Okay.
>> 
>>> Could we enable the bogon list as a default for dial-up interfaces in
>>> IPFire 3.x ?
>> 
>> Not only dial-up, but this probably would not be a dynamic list, but
>> rather a substantial part of the firewall.
> ACK.
> 
> Thanks, and best regards,
> Peter Müller
> -- 
> Microsoft DNS service terminates abnormally when it recieves a response
> to a DNS query that was never made.  Fix Information: Run your DNS
> service on a different platform.
> 		-- bugtraq

Re: Status emails and IP Blocklists

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 8058 bytes --]

Hi Tim,

> On 31 Mar 2019, at 23:57, Tim FitzGeorge <ipfr(a)tfitzgeorge.me.uk> wrote:
> 
> Hi,
> 
> After picking up some unexpected jobs and then loosing my internet connection, I'm now just about ready to submit the patches for status emails (it's difficult to work on something that's meant to download from the internet if you haven't got a working connection).  I thought it would probably be a good idea to give some warning before sending the patches.

I cannot really tell if this is an overall good or bad thing, but happy to have you back!

> This will be for the status emails; sending (optionally GPG encrypted) emails giving information about the system on a user defined schedule.

Thanks for the heads up. Feel free to send an RFC or so if you are unsure and so on.

Best,
-Michael

> 
> Tim
> 
> On 01/12/2018 19:46, Michael Tremer wrote:
>> Hey,
>> 
>> 
>>> On 1 Dec 2018, at 18:20, Tim FitzGeorge <ipfr(a)tfitzgeorge.me.uk>
>>>  wrote:
>>> 
>>> Hi,
>>> 
>>> On 30/11/2018 11:04, Michael Tremer wrote:
>>> 
>>>> Hey Tim,
>>>> 
>>>> thanks for your email!
>>>> 
>>>> Those addons look great. Quite neat and tidy code and probably they are
>>>> scratching an itch for some people.
>>>> 
>>>> On Thu, 2018-11-29 at 21:11 +0000, Tim FitzGeorge wrote:
>>>> 
>>>>> I've written a couple of addons for my installations of IPFire.  They're
>>>>> available on github and some other people have tried them; they seem to
>>>>> be fairly well received and it's been suggested that it may be worth
>>>>> making them available through pakfire as official addons.
>>>>> 
>>>> Where did you publish them before?
>>>> 
>>> I've not published them before - I didn't even announce them on the
>>> forums, but someone must have looked around after looking at the IDS
>>> rule updater.
>>> 
>> Well, I guess great software finds its users on its own...
>> 
>> 
>>>>> The first addon provides the ability to send status emails. You can
>>>>> define multiple schedules and the items to be included in each email. 
>>>>> By choosing parameters carefully it's possible to get it to send emails
>>>>> on some error conditions.  The emails can be encrypted with GPG.  The
>>>>> architecture makes it easy to add further items to be reported on.
>>>>> 
>>>> Could you send an example email what it looks like? I do not see any reason why
>>>> this should not be part of the distribution and would like to ask you to submit
>>>> this as a patch that can be merged into mainline.
>>>> 
>>> I've attached a jpeg of the HTML version of a test email.  It's had
>>> certain information redacted.  I don't include quite so much information
>>> in my normal reports.  It's also capable of some additional information
>>> (for example errors) which only show up when necessary.
>>> 
>> Wow this is a lot. As in an overwhelming amount of graphs and data.
>> 
>> I am not sure if this is useful when its altogether, but I guess that can be
>> decided by each user…
>> 
>> About the UI: I guess that could be a lot shorter. I find it quite logical
>> that when someone wants a weekly report, the graphs should show the whole
>> week and not only the last day. So that can be a single switch that makes
>> many of the other options further done redundant.
>> 
>> 
>>> The text emails can contain everything except the graphs.  I've got my
>>> systems set up to send an HTML email at midnight with a summary of the
>>> previous day's information including some graphs, plus a text email of
>>> error conditions every hour - this only gets sent if there are errors.
>>> 
>>> 
>>> This one just turned up:
>>> 
>>> Error check report
>>> 
>>> 
>>> System
>>> ------
>>> 
>>>  SSH
>>> 
>>>    Logins
>>> 
>>>    User       From        Count
>>>    root  192.168.999.999    2
>>> 
>>> 
>>> I'll start working on a patch.  I think my one question at this point is
>>> where should it go in the menus?  I put it under 'IPFire' since that
>>> seems to be where miscellaneous addons go, but is there a better place
>>> for it?
>>> 
>> Good questions. I am not very happy with the IPFire sub-menu because there
>> is no point in it. This is a left-over from about 15 years ago when we used
>> IPCop as a base.
>> 
>> I think this could even be part of the email settings CGI; or it should
>> go into logging.
>> 
>> 
>>>> Maybe we can extend this over time and have it send more information if there
>>>> are any requests.
>>>> 
>>> Yes, it's got a plug-in architecture and in most cases adding more
>>> information is quite easy.  The main code takes care of formatting,
>>> whether for HTML or Text, so a table can be added with one function call
>>> which is passed an array of arrays.
>>> 
>>>> Would you be up for maintaining this long-term?
>>>> 
>>> Yes.
>>> 
>> Great!
>> 
>> 
>>>> Did you develop this for yourself or for work or has this been sponsored by
>>>> someone else?
>>>> 
>>> I did it for myself.  As well as my home system, I've got another one
>>> set up at a small charity, and I wanted a way to see its status without
>>> having to go over there.  I didn't want to set up a VPN just for logging
>>> in and checking status.
>>> 
>> Looks like a lot of work as a workaround to not set up a VPN.
>> 
>> 
>>>>> The second addon handles the setting up and updating of IP Address
>>>>> Blocklists in the firewall.  It includes options to select which lists
>>>>> to use, and some control over how frequently to check for updates.
>>>>> 
>>>> I guess Peter might be quite excited about this :)
>>>> 
>>>> I personally do not have much use for this, but again, why should this not
>>>> become part of IPFire?
>>>> 
>>>> I did not install any of these yet, so could you maybe excuse lazy me and send
>>>> screenshots? :)
>>>> 
>>> Attached.  The WUI for this is fairly simple.  There's also a logwatch
>>> plug in so that a summary of the update status appears in the log summary.
>>> 
>> See my comments above. I also have some other probably minor questions regarding
>> some things on here, but I guess that can wait…
>> 
>> 
>>>>> Both include WUI pages for configuration and language files.  They're
>>>>> fully functional, but would require some checking and minor updates. 
>>>>> The source can be seen at 
>>>>> https://github.com/timfprogs
>>>>>  .
>>>>> 
>>>> I have seen a third one which updates Snort rules. I am sure that you have heard
>>>> about us changing to suricata soon (test images are available). However, the
>>>> rules are roughly the same and the same update tools can be used. So, again,
>>>> would you be interested to have this in the distribution and maintain it?
>>>> 
>>> Definitely.  I believe that there's already an automatic updater
>>> provided, but I think mine has more facilities.  I'm planning to install
>>> the suricata test image in the next few weeks and have a good look at it.
>>> 
>> Yes, we should work on one thing after the other. Great that you join testing.
>> 
>> Potentially we should think about working on this first now, so that suricata
>> can go out as soon as possible with as many features as possible.
>> 
>> Would you be okay with that?
>> 
>> 
>>>>> I'm aware that there other people have made addons for both these
>>>>> purposes, which maybe suggests that it's functionality that is worth adding.
>>>>> 
>>>> Best,
>>>> -Michael
>>>> 
>>>> P.S. Did you get any help building these or do you speak four languages?
>>>> 
>>> Alas, I only really speak English (although I do have some limited
>>> knowledge of French and Latin).  I used Google translate, so I expect
>>> some errors - hopefully amusing ones rather than insulting.
>>> 
>> Good question. I have no idea. We can check with a speaker of any of those
>> languages or ship it English-only.
>> 
>> Best,
>> -Michael
>> 
>> 
>>> 
>>> Tim
>>> 
>>> 
>>> 
>>> <statusmail_email.jpeg><statusmail_wui.jpeg><blocklist-wui.png>
>>> 
>> 
> 
> 

Re: Status emails and IP Blocklists

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 6637 bytes --]

Hey,

> On 1 Dec 2018, at 18:20, Tim FitzGeorge <ipfr(a)tfitzgeorge.me.uk> wrote:
> 
> Hi,
> 
> On 30/11/2018 11:04, Michael Tremer wrote:
>> Hey Tim,
>> 
>> thanks for your email!
>> 
>> Those addons look great. Quite neat and tidy code and probably they are
>> scratching an itch for some people.
>> 
>> On Thu, 2018-11-29 at 21:11 +0000, Tim FitzGeorge wrote:
>>> I've written a couple of addons for my installations of IPFire.  They're
>>> available on github and some other people have tried them; they seem to
>>> be fairly well received and it's been suggested that it may be worth
>>> making them available through pakfire as official addons.
>> Where did you publish them before?
> I've not published them before - I didn't even announce them on the
> forums, but someone must have looked around after looking at the IDS
> rule updater.

Well, I guess great software finds its users on its own...

>>> The first addon provides the ability to send status emails. You can
>>> define multiple schedules and the items to be included in each email. 
>>> By choosing parameters carefully it's possible to get it to send emails
>>> on some error conditions.  The emails can be encrypted with GPG.  The
>>> architecture makes it easy to add further items to be reported on.
>> Could you send an example email what it looks like? I do not see any reason why
>> this should not be part of the distribution and would like to ask you to submit
>> this as a patch that can be merged into mainline.
> 
> I've attached a jpeg of the HTML version of a test email.  It's had
> certain information redacted.  I don't include quite so much information
> in my normal reports.  It's also capable of some additional information
> (for example errors) which only show up when necessary.

Wow this is a lot. As in an overwhelming amount of graphs and data.

I am not sure if this is useful when its altogether, but I guess that can be
decided by each user…

About the UI: I guess that could be a lot shorter. I find it quite logical
that when someone wants a weekly report, the graphs should show the whole
week and not only the last day. So that can be a single switch that makes
many of the other options further done redundant.

> The text emails can contain everything except the graphs.  I've got my
> systems set up to send an HTML email at midnight with a summary of the
> previous day's information including some graphs, plus a text email of
> error conditions every hour - this only gets sent if there are errors.
> 
> 
> This one just turned up:
> 
> Error check report
> 
> 
> System
> ------
> 
>  SSH
> 
>    Logins
> 
>    User       From        Count
>    root  192.168.999.999    2
> 
> 
> I'll start working on a patch.  I think my one question at this point is
> where should it go in the menus?  I put it under 'IPFire' since that
> seems to be where miscellaneous addons go, but is there a better place
> for it?

Good questions. I am not very happy with the IPFire sub-menu because there
is no point in it. This is a left-over from about 15 years ago when we used
IPCop as a base.

I think this could even be part of the email settings CGI; or it should
go into logging.

>> 
>> Maybe we can extend this over time and have it send more information if there
>> are any requests.
> Yes, it's got a plug-in architecture and in most cases adding more
> information is quite easy.  The main code takes care of formatting,
> whether for HTML or Text, so a table can be added with one function call
> which is passed an array of arrays.
>> 
>> Would you be up for maintaining this long-term?
> Yes.

Great!

>> 
>> Did you develop this for yourself or for work or has this been sponsored by
>> someone else?
> 
> I did it for myself.  As well as my home system, I've got another one
> set up at a small charity, and I wanted a way to see its status without
> having to go over there.  I didn't want to set up a VPN just for logging
> in and checking status.

Looks like a lot of work as a workaround to not set up a VPN.

>>> The second addon handles the setting up and updating of IP Address
>>> Blocklists in the firewall.  It includes options to select which lists
>>> to use, and some control over how frequently to check for updates.
>> I guess Peter might be quite excited about this :)
>> 
>> I personally do not have much use for this, but again, why should this not
>> become part of IPFire?
>> 
>> I did not install any of these yet, so could you maybe excuse lazy me and send
>> screenshots? :)
> Attached.  The WUI for this is fairly simple.  There's also a logwatch
> plug in so that a summary of the update status appears in the log summary.

See my comments above. I also have some other probably minor questions regarding
some things on here, but I guess that can wait…

>>> Both include WUI pages for configuration and language files.  They're
>>> fully functional, but would require some checking and minor updates. 
>>> The source can be seen at https://github.com/timfprogs .
>> I have seen a third one which updates Snort rules. I am sure that you have heard
>> about us changing to suricata soon (test images are available). However, the
>> rules are roughly the same and the same update tools can be used. So, again,
>> would you be interested to have this in the distribution and maintain it?
> Definitely.  I believe that there's already an automatic updater
> provided, but I think mine has more facilities.  I'm planning to install
> the suricata test image in the next few weeks and have a good look at it.

Yes, we should work on one thing after the other. Great that you join testing.

Potentially we should think about working on this first now, so that suricata
can go out as soon as possible with as many features as possible.

Would you be okay with that?

>>> I'm aware that there other people have made addons for both these
>>> purposes, which maybe suggests that it's functionality that is worth adding.
>> Best,
>> -Michael
>> 
>> P.S. Did you get any help building these or do you speak four languages?
> 
> Alas, I only really speak English (although I do have some limited
> knowledge of French and Latin).  I used Google translate, so I expect
> some errors - hopefully amusing ones rather than insulting.

Good question. I have no idea. We can check with a speaker of any of those
languages or ship it English-only.

Best,
-Michael

> 
> 
> Tim
> 
> 
> 
> <statusmail_email.jpeg><statusmail_wui.jpeg><blocklist-wui.png>

Status emails and IP Blocklists

[Tim FitzGeorge]

[-- Attachment #1: Type: text/plain, Size: 1182 bytes --]

I've written a couple of addons for my installations of IPFire.  They're
available on github and some other people have tried them; they seem to
be fairly well received and it's been suggested that it may be worth
making them available through pakfire as official addons.


The first addon provides the ability to send status emails. You can
define multiple schedules and the items to be included in each email. 
By choosing parameters carefully it's possible to get it to send emails
on some error conditions.  The emails can be encrypted with GPG.  The
architecture makes it easy to add further items to be reported on.


The second addon handles the setting up and updating of IP Address
Blocklists in the firewall.  It includes options to select which lists
to use, and some control over how frequently to check for updates.


Both include WUI pages for configuration and language files.  They're
fully functional, but would require some checking and minor updates. 
The source can be seen at https://github.com/timfprogs .


I'm aware that there other people have made addons for both these
purposes, which maybe suggests that it's functionality that is worth adding.