Re: Next steps of IPFire Location

[Stefan Schantl <stefan.schantl@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 2651 bytes --]

Hello Michael,

I've installed the latest Core 153 from unstable Branch and benchmarked
the location database export on my IPfire Mini Appliance.

[root(a)gate ~]# time location export --directory=/tmp --format=xt_geoip

real    0m53.629s
user    0m52.144s
sys     0m0.702s

Best regards,

-Stefan
> I managed to export the database on my IPFire Mini Appliance in about
> one minute:
> 
> [root(a)fw01 libloc]# time location export --directory=tmp --
> format=xt_geoip
> 
> real    1m3.423s
> user    1m1.470s
> sys     0m1.009s
> 
> The system was under a little bit of load due to some network
> activity, but I am generally quite happy with this.
> 
> I added some more fine-tuning and could decrease the export time on
> my Debian test machine by another 40% to around six seconds.
> 
> -Michael
> 
> > On 18 Nov 2020, at 19:26, Michael Tremer <michael.tremer(a)ipfire.org
> > > wrote:
> > 
> > Hello,
> > 
> > Since there are now so many people involved in this project, I
> > would like to send a little email to report what has happened and
> > what the next steps are going to be…
> > 
> > I have spent (unfortunately) a lot of time to add the ability that
> > we can “flatten the tree” when we export it. That means that no
> > network will overlap with another one and we do not need to care
> > about any orders of iptables rules being added.
> > 
> > That was quite a slow process in Python and since IPFire has to run
> > on small hardware it needed to be ported into C. The tree can now
> > be exported in about 10 seconds on my development box which is
> > running an Intel Xeon E5-2630 processor.
> > 
> > I hope that we can reach acceptable times of only a few minutes on
> > the slowest systems and way under a minute on an average system. To
> > find out about that, I need you help:
> > 
> > I just pushed all changes into the next tree and would everyone who
> > can to install that and report how long this command runs:
> > 
> >  mkdir tmp
> >  time location export --directory=tmp --format=xt_geoip
> > 
> > It would also be great if you could all check the output if this is
> > still correct. I have done what I could, but I want to have an
> > extra pair of eyes if possible.
> > 
> > If this works well, and no more regressions are being found, I will
> > tag this release and we will close Core Update 153.
> > 
> > We will then roll out all recent changes that make the database
> > better, but had to be held back because the tree needs to be
> > flattened for most use-cases now. All older clients will then no
> > longer receive an updated database any more.
> > 
> > Happy testing.
> > 
> > Best,
> > -Michael