Re: fireperf results

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 4297 bytes --]

Hello Adolf,

This is very surprising to me. I am almost shocked.

Maybe any of my assumptions are wrong, but if this is the actual throughput of this piece of hardware, I find this not enough.

> On 16 Feb 2021, at 12:44, Adolf Belka (ipfire-dev) <adolf.belka(a)ipfire.org> wrote:
> 
> Hi All,
> 
> Following are the fireperf results I obtained:-
> 
> server: IPFire 2.25 - Core Update 153; Intel Celeron CPU J1900 @ 1.99GHz x4; I211 Gigabit Network Connection

You have a small processor here with a rather high clock rate. Four cores at 2 GHz is quite something.

However, it is a Celeron processor and that means it is a bit more stripped down than others. Usually it is caches and pipeline throughput. It might be that, that bites you really bad here.

You have a better than average NIC. The Intel network controllers are not bad, although the i2xx series is not fully active.

Could you please send the output of “cat /proc/interrupts” so that we can see how many queues they have?

> client: Arch Linux; Intel Core i5-8400 CPU @ 2.80GHz 6 core; 1GBit nic
> 
> Server:
> fireperf -s -P 10000 -p 63000:630010
> 
> 
> Client:
> fireperf -c <IP address> -P 1 -x -p 63000:63010 -> 100 - 3000 cps strongly fluctuating. After a couple of minutes the client cps went down to 0 and stayed there. I had to stop fireperf and restart the terminal to get it working again.
> 
> fireperf -c <IP address> -P 10 -x -p 63000:63010 ->250 - 500 cps fluctuating
> 
> fireperf -c <IP address> -P 100 -x -p 63000:63010 -> 220 - 1000 cps fluctuating
> 
> fireperf -c <IP address> -P 1000 -x -p 63000:63010 -> 1200 - 2500 cps fluctuating
> 
> fireperf -c <IP address> -P 10000 -x -p 63000:63010 -> 0 - 7000 cps hugely fluctuating

>From the beginning you have quite a large fluctuation here. Some is normal, but this is a lot. It seems that the system is overloaded from the very beginning.

I have not done experiments with lots of different hardware (used the same usually), but Daniel has, and we normally have the systems being very idle with only one connection at a time. There isn’t too much to do for the CPU except waiting.

> In all cases the cpu utilisation was quite low on both IPFire and the Arch Linux desktop.

Not surprising on the desktop side, because there wasn’t a lot stuff to do.

> I then repeated the above tests removing the -x option so I could see the data bandwidth.
> 
> 
> fireperf -c <IP address> -P 1 -p 63000:63010 -> 225Mb/s - 1 core at 100%, rest around 30% to 40%

This is the most surprising part.

The IPFire Mini Appliance for example only has 1 GHz of clock and it doesn’t have any problems with transmissing a whole gigabit a second of data. This system has double the clock speed and the same NIC (or at least very similar to it).

> fireperf -c <IP address> -P 10 -p 63000:63010 -> 185Mb/s - similar as above
> 
> fireperf -c <IP address> -P 100 -p 63000:63010 -> 210Mb/s - similar to above

The bandwidth should have increased here. That means we know that the bottleneck is not the network, but something else.

The one core that is maxed out is to some good extend the fireperf process generating packets. The rest is overhead of the OS, network stack and NIC driver. Which feels way too high for me.

> fireperf -c <IP address> -P 1000 -p 63000:63010 -> 370 - 450Mb/s - 2 cores at 100%, rest at 30% to 40%
> 
> fireperf -c <IP address> -P 10000 -p 63000:63010 -> 400Mb/s - 1Gb/s - 2 cores at 100%, rest at 40% to 50%

You seem to have more than one receive queue as it looks like.

Did you actually achieve the 10k connections?

> I recently got my Glass Fibre Gigabit connection connected. The supplier hooked his laptop directly to the media converter and got around 950Mb/s

Could you test “speedtest-cli” and see what that reports?

> Using the same speed test as he used but going through my IPFire hardware I get around 225Mb/s.
> 
> Although my hardware has four Intel I211 Gigabit NIC's, I have suspected that their performance is limited by the processor.

It sounds like it. Lets see what more information we can gather and hopefully find it.

Can you run powertop along the bechmark and see what that says?

-Michael

> Regards,
> 
> Adolf.
> 
>