From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: fireperf results Date: Tue, 16 Feb 2021 16:16:36 +0000 Message-ID: In-Reply-To: <4e13b606-acf2-68c6-6fc2-972de695e80a@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0740806840754124311==" List-Id: --===============0740806840754124311== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello Adolf, This is very surprising to me. I am almost shocked. Maybe any of my assumptions are wrong, but if this is the actual throughput o= f this piece of hardware, I find this not enough. > On 16 Feb 2021, at 12:44, Adolf Belka (ipfire-dev) wrote: >=20 > Hi All, >=20 > Following are the fireperf results I obtained:- >=20 > server: IPFire 2.25 - Core Update 153; Intel Celeron CPU J1900 @ 1.99GHz x4= ; I211 Gigabit Network Connection You have a small processor here with a rather high clock rate. Four cores at = 2 GHz is quite something. However, it is a Celeron processor and that means it is a bit more stripped d= own than others. Usually it is caches and pipeline throughput. It might be th= at, that bites you really bad here. You have a better than average NIC. The Intel network controllers are not bad= , although the i2xx series is not fully active. Could you please send the output of =E2=80=9Ccat /proc/interrupts=E2=80=9D so= that we can see how many queues they have? > client: Arch Linux; Intel Core i5-8400 CPU @ 2.80GHz 6 core; 1GBit nic >=20 > Server: > fireperf -s -P 10000 -p 63000:630010 >=20 >=20 > Client: > fireperf -c -P 1 -x -p 63000:63010 -> 100 - 3000 cps strongly = fluctuating. After a couple of minutes the client cps went down to 0 and stay= ed there. I had to stop fireperf and restart the terminal to get it working a= gain. >=20 > fireperf -c -P 10 -x -p 63000:63010 ->250 - 500 cps fluctuating >=20 > fireperf -c -P 100 -x -p 63000:63010 -> 220 - 1000 cps fluctua= ting >=20 > fireperf -c -P 1000 -x -p 63000:63010 -> 1200 - 2500 cps fluct= uating >=20 > fireperf -c -P 10000 -x -p 63000:63010 -> 0 - 7000 cps hugely = fluctuating >>From the beginning you have quite a large fluctuation here. Some is normal, b= ut this is a lot. It seems that the system is overloaded from the very beginn= ing. I have not done experiments with lots of different hardware (used the same us= ually), but Daniel has, and we normally have the systems being very idle with= only one connection at a time. There isn=E2=80=99t too much to do for the CP= U except waiting. > In all cases the cpu utilisation was quite low on both IPFire and the Arch = Linux desktop. Not surprising on the desktop side, because there wasn=E2=80=99t a lot stuff = to do. > I then repeated the above tests removing the -x option so I could see the d= ata bandwidth. >=20 >=20 > fireperf -c -P 1 -p 63000:63010 -> 225Mb/s - 1 core at 100%, r= est around 30% to 40% This is the most surprising part. The IPFire Mini Appliance for example only has 1 GHz of clock and it doesn=E2= =80=99t have any problems with transmissing a whole gigabit a second of data.= This system has double the clock speed and the same NIC (or at least very si= milar to it). > fireperf -c -P 10 -p 63000:63010 -> 185Mb/s - similar as above >=20 > fireperf -c -P 100 -p 63000:63010 -> 210Mb/s - similar to above The bandwidth should have increased here. That means we know that the bottlen= eck is not the network, but something else. The one core that is maxed out is to some good extend the fireperf process ge= nerating packets. The rest is overhead of the OS, network stack and NIC drive= r. Which feels way too high for me. > fireperf -c -P 1000 -p 63000:63010 -> 370 - 450Mb/s - 2 cores = at 100%, rest at 30% to 40% >=20 > fireperf -c -P 10000 -p 63000:63010 -> 400Mb/s - 1Gb/s - 2 cor= es at 100%, rest at 40% to 50% You seem to have more than one receive queue as it looks like. Did you actually achieve the 10k connections? > I recently got my Glass Fibre Gigabit connection connected. The supplier ho= oked his laptop directly to the media converter and got around 950Mb/s Could you test =E2=80=9Cspeedtest-cli=E2=80=9D and see what that reports? > Using the same speed test as he used but going through my IPFire hardware I= get around 225Mb/s. >=20 > Although my hardware has four Intel I211 Gigabit NIC's, I have suspected th= at their performance is limited by the processor. It sounds like it. Lets see what more information we can gather and hopefully= find it. Can you run powertop along the bechmark and see what that says? -Michael > Regards, >=20 > Adolf. >=20 >=20 --===============0740806840754124311==--