From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: Next steps of IPFire Location Date: Mon, 23 Nov 2020 11:15:56 +0000 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4800075550052829726==" List-Id: --===============4800075550052829726== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Thank you for all your feedback. It helped a lot. Or rather=E2=80=A6 It would have if Arne didn=E2=80=99t find that this whole = algorithm doesn=E2=80=99t work :) It simply does not do what we expect it to do and after my fix this takes a l= ong time again. I hope that I will find some time this week to improve the implementation fur= ther and will get back to you all very soon :) Best, -Michael > On 22 Nov 2020, at 20:06, Adolf Belka wrote: >=20 > Dear All, >=20 >=20 > I have the results now for the location testing on my Intel(R) Celeron(R) C= PU J1900 @ 1.99GHz production box. >=20 > Stable > real 0m37.240s > user 0m37.086s > sys 0m0.070s >=20 >=20 > Unstable > real 0m25.827s > user 0m25.568s > sys 0m0.123s >=20 >=20 > Regards, >=20 > Adolf. >=20 > On 19/11/2020 18:48, Adolf Belka wrote: >> I have tested it out on my IPFire virtual machine test bed. >>=20 >> I got between 4.8 and 5 secs. >>=20 >> The computer is an i5-8400 CPU @ 2.80GHz >>=20 >> I will also be testing it out on my production box which is a Intel(R) Cel= eron(R) CPU J1900 @ 1.99GHz but I have to prepare for that as I will need to = re-install Core152 after carrying out the test. Will email the results when I= have them. >>=20 >> Regards, >> Adolf. >>=20 >>=20 >> On 19/11/2020 14:10, Michael Tremer wrote: >>> I managed to export the database on my IPFire Mini Appliance in about one= minute: >>>=20 >>> [root(a)fw01 libloc]# time location export --directory=3Dtmp --format=3Dx= t_geoip >>>=20 >>> real 1m3.423s >>> user 1m1.470s >>> sys 0m1.009s >>>=20 >>> The system was under a little bit of load due to some network activity, b= ut I am generally quite happy with this. >>>=20 >>> I added some more fine-tuning and could decrease the export time on my De= bian test machine by another 40% to around six seconds. >>>=20 >>> -Michael >>>=20 >>>> On 18 Nov 2020, at 19:26, Michael Tremer w= rote: >>>>=20 >>>> Hello, >>>>=20 >>>> Since there are now so many people involved in this project, I would lik= e to send a little email to report what has happened and what the next steps = are going to be=E2=80=A6 >>>>=20 >>>> I have spent (unfortunately) a lot of time to add the ability that we ca= n =E2=80=9Cflatten the tree=E2=80=9D when we export it. That means that no ne= twork will overlap with another one and we do not need to care about any orde= rs of iptables rules being added. >>>>=20 >>>> That was quite a slow process in Python and since IPFire has to run on s= mall hardware it needed to be ported into C. The tree can now be exported in = about 10 seconds on my development box which is running an Intel Xeon E5-2630= processor. >>>>=20 >>>> I hope that we can reach acceptable times of only a few minutes on the s= lowest systems and way under a minute on an average system. To find out about= that, I need you help: >>>>=20 >>>> I just pushed all changes into the next tree and would everyone who can = to install that and report how long this command runs: >>>>=20 >>>> mkdir tmp >>>> time location export --directory=3Dtmp --format=3Dxt_geoip >>>>=20 >>>> It would also be great if you could all check the output if this is stil= l correct. I have done what I could, but I want to have an extra pair of eyes= if possible. >>>>=20 >>>> If this works well, and no more regressions are being found, I will tag = this release and we will close Core Update 153. >>>>=20 >>>> We will then roll out all recent changes that make the database better, = but had to be held back because the tree needs to be flattened for most use-c= ases now. All older clients will then no longer receive an updated database a= ny more. >>>>=20 >>>> Happy testing. >>>>=20 >>>> Best, >>>> -Michael >>>=20 --===============4800075550052829726==--