Hello,
Because I have been quiet lately I wanted to update everyone a little bit on what I have been working on in the last couple of weeks.
Nothing is a secret, but I have been toying around with a couple of things and they are simply not done, yet, so there is nothing to show, but I still wanted to keep everyone up to date.
Project #1: Benchmarks
Together with Daniel, I took an entire day to test performance of our appliances. This is something we have done before, but we wanted to know what has changed over the recent software changes, especially with the new kernel. And we have discovered some amazing things that the community seems to have generally confirmed on other hardware, too: The new kernel is a lot faster.
We spent most time on the IPFire Mini Appliance, because it is (obviously) the slowest, but also the most commonly bought model. There is a lot of potential here from which plenty of people would benefit.
I do not have my notes with me now while writing this email, but generally, we can saturate all interfaces now rather easily using fireperf and we could bring up the throughput of the IPS from only a few mbps to about 120 MBit/s (duplex). This is a little bit of a mile stone because that makes this appliances suitable for a 100 MBit/s connection which it wasn’t really before.
We have also experimented a lot with offloading features for network interfaces so that the OS has less work to do which will result in a couple of changes soon.
There is a lot more we can do to increase throughput even more (see below).
We will soon write up all these benchmark results and put them on the wiki, although we already have lots of patches that will change them again, but we would like to track progress of how much we are improving IPFire.
Project #2: IPFire on AWS for ARM
IPFire will soon be available on AWS Graviton 2 instances. The code is already done, but the entire product is under review with AWS right now.
This going to make IPFire available on a non-Intel/non-AMD-based platform which is exciting and new and also cheaper. I have a little blog post prepared for the release of it and it will strengthen IPFire in the cloud.
Project #3: Cake
The new kernel is coming with a couple of new features that we can use now and so I started playing around with Cake:
https://www.bufferbloat.net/projects/codel/wiki/Cake/
I only changed the “out-of-the box” configuration and did not touch the QoS, yet (lack of time):
https://git.ipfire.org/?p=people/ms/ipfire-2.x.git;a=commitdiff;h=dbebf0703e...
I have also disabled any extra qdiscs on interfaces where it probably does not make too much sense (bridges, GRE/VTI, …). That should save us a couple of CPU cycles again.
If anyone wants to deploy this on their systems, I would be happy to hear some feedback.
Project #4: Making the IPS 100x faster
Then, I have been working with Stefan on making the IPS faster. In our benchmarks, we discovered, that we spent most CPU cycles on copying the data packets from the kernel to the user space. That is a very expensive job and has only gotten worse since we have the Spectre/Meltdown fixes everywhere. However, when the IPS is scanning packets, it cannot always do that very well.
For example a TLS connection can be analysed during the handshake, but any traffic coming after that is practically random noise. It is encrypted and one of the goals of TLS is to not give anyone any clue about what might be in the data packet. So why would we send such a packet to suricata in the first place? We shouldn’t.
And that is what we have been working on which makes the IPS so much faster because we cut out the bad part that is passing dumb traffic for no reason.
We have experimental patches for this, but there is a lot that needs to change for this to work and co-exist with other features that mangle packets, specifically the QoS and NAT. We are working on this and hopefully will be able to send a tech preview soon.
Yeah, so this is what I have in the pipeline.
FYI. Feedback is of course greatly appreciated.
Best, -Michael