From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: location@lists.ipfire.org Subject: Re: Auto-generated plaintext updates; re: regression Date: Thu, 10 Mar 2022 14:48:40 +0000 Message-ID: <7433146A-A01A-49CE-A54C-AE5E33372D12@ipfire.org> In-Reply-To: <9d5cae24-744d-cdc6-fd69-75876344072a@posteo.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5710713412110282735==" List-Id: --===============5710713412110282735== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, > On 10 Mar 2022, at 14:45, Jordan Savoca wrote: >=20 > On 3/10/22 02:15, Michael Tremer wrote: >> Yes, I will do this as soon as possible. At the moment, there are lots of = other changes in the repository and I didn=E2=80=99t yet know for certain tha= t they won=E2=80=99t break anything. >=20 > Awesome, thank you! >=20 >> But in general, I would like to say, that you should not use the text dump= . It is only there for audit purposes and committed to the Git repository so = that everyone can see any changes to the database easily. >> I would strongly recommend using the binary database and the Python/Perl b= indings. What is your application for the data? >=20 > I had originally intended to use the binary database but I found the non-st= andard format and introduction of dependencies less flexible for my use case = relative to fetching the plaintext set via submodule to store in a sqlite db = where it can then be queried using standard libraries. For now my use case is= limited to visualizing announcement changes over time but I hope to make som= ething of import at some point. I would strongly recommend this, because parsing the text database and going = with the first match is quite likely going to give you inaccurate results. The reason why we are going with a binary format is because it is organised a= s a binary tree and therefore can be searched *very* quickly. It also allows = us to store multiple networks with the same start address, but a different pr= efix length which is what you won=E2=80=99t have in a relational database and= therefore you will quite likely have inaccurate results. We kept the library as small as possible and we are looking at upstreaming it= to all major distributions because that will help integrating it into other = software a lot easier. Any help with this would be greatly appreciated. > The semi-recent change to maxmind's license + requiring an account to fetch= geolocation data led me to look into alternative sources, so thank you for y= our work on location(8) :). You are welcome :) You can also support us with a donation if you like at htt= ps://www.ipfire.org/donate. Best, -Michael >=20 > --=20 > Jordan Savoca --===============5710713412110282735==--