From mboxrd@z Thu Jan  1 00:00:00 1970
From: Michael Tremer <michael.tremer@ipfire.org>
To: location@lists.ipfire.org
Subject: Re: What generates database.db from database.txt?
Date: Tue, 20 Oct 2020 17:08:19 +0100
Message-ID: <0A556AF2-9F53-467C-9627-1208A6E20CAA@ipfire.org>
In-Reply-To: <4eaccd86-f9d9-d53d-738a-6103179a15ef@gmail.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============3592746972323882878=="
List-Id: <location.lists.ipfire.org>

--===============3592746972323882878==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Hello Gisle,

Thank you for your email.

> On 19 Oct 2020, at 16:53, Gisle Vanem <gisle.vanem(a)gmail.com> wrote:
>=20
> Hello list, my ports post.
>=20
> I'm on Windows and have little interest in installing
> IPFire or do any Linux related. But the 'libloc'
> library caught my interest. So I did a port to
> MSVC, MinGW and clang-cl (x86 only) in a hurry.

That sounds great!

We are looking for someone with some experience with Windows to port libloc. =
It should not be super complicated, but since I am not the expert I am not to=
o sure.

Would you be willing to tidy up your code (if required) and submit them on he=
re so that we can officially support Windows?

> Running all the 'test-*' programs, it worked seemingly
> well. Until I built the Python3 module and played with
> e.g. 'location dump' and 'verify'. Nothing worked :-(

What didn=E2=80=99t work specifically? Reading and writing the database?

> So I reckon the .db file is hosed somehow. AFAICS, it's
> not an issue with structure packing or 'fopen(path, "r")'.

Hmm=E2=80=A6 it would be interesting to debug that.

> Anyway, I want to regenerate 'database.db' from 'database.txt'
> 'countries.txt' and the 'override' files myself. But how?

We do not generate the database from database.txt. database.txt is being gene=
rated from the database. The text dump is just there so that we have a (sort =
of) human-readable version which we can diff against the previous one to manu=
ally verify any changes.

What you will need to import the raw data is location-importer which comes wi=
th libloc and a PostgreSQL server.

location-importer =E2=80=94help will show you how you pass the database crede=
ntials.

On our servers we regularly run =E2=80=9Clocation-import update-overrides fil=
e-a.txt file-b.txt=E2=80=9D, then "location-importer update-whois=E2=80=9D, t=
hen we run =E2=80=9Clocation-importer update-announcements=E2=80=9D which con=
nects to a locally running instance of Bird with a full BGP feed to import th=
ose.

Finally we run =E2=80=9Clocation-importer write =E2=80=A6 database.db=E2=80=
=9D (fill in the gaps with your vendor name, description, key, etc.) and fina=
lly you can run =E2=80=9Clocation -d database.db dump=E2=80=9D to export it i=
nto the text format.

Generating the whole database might take a couple of hours depending on how b=
eefy your machine is.

Let me know if this works for you.

> According to the git log at:
>  git://git.ipfire.org/location/location-database.git
>=20
> there once was a 'compile-database' script. Not now.

Yes, this has all been moved into =E2=80=9Clocation-importer=E2=80=9D so that=
 we separate code and data in two different Git repositories.

Best,
-Michael

>=20
> --=20
> --gv


--===============3592746972323882878==--