Re: What generates database.db from database.txt?

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 2653 bytes --]

Hello Gisle,

Thank you for your email.

> On 19 Oct 2020, at 16:53, Gisle Vanem <gisle.vanem(a)gmail.com> wrote:
> 
> Hello list, my ports post.
> 
> I'm on Windows and have little interest in installing
> IPFire or do any Linux related. But the 'libloc'
> library caught my interest. So I did a port to
> MSVC, MinGW and clang-cl (x86 only) in a hurry.

That sounds great!

We are looking for someone with some experience with Windows to port libloc. It should not be super complicated, but since I am not the expert I am not too sure.

Would you be willing to tidy up your code (if required) and submit them on here so that we can officially support Windows?

> Running all the 'test-*' programs, it worked seemingly
> well. Until I built the Python3 module and played with
> e.g. 'location dump' and 'verify'. Nothing worked :-(

What didn’t work specifically? Reading and writing the database?

> So I reckon the .db file is hosed somehow. AFAICS, it's
> not an issue with structure packing or 'fopen(path, "r")'.

Hmm… it would be interesting to debug that.

> Anyway, I want to regenerate 'database.db' from 'database.txt'
> 'countries.txt' and the 'override' files myself. But how?

We do not generate the database from database.txt. database.txt is being generated from the database. The text dump is just there so that we have a (sort of) human-readable version which we can diff against the previous one to manually verify any changes.

What you will need to import the raw data is location-importer which comes with libloc and a PostgreSQL server.

location-importer —help will show you how you pass the database credentials.

On our servers we regularly run “location-import update-overrides file-a.txt file-b.txt”, then "location-importer update-whois”, then we run “location-importer update-announcements” which connects to a locally running instance of Bird with a full BGP feed to import those.

Finally we run “location-importer write … database.db” (fill in the gaps with your vendor name, description, key, etc.) and finally you can run “location -d database.db dump” to export it into the text format.

Generating the whole database might take a couple of hours depending on how beefy your machine is.

Let me know if this works for you.

> According to the git log at:
>  git://git.ipfire.org/location/location-database.git
> 
> there once was a 'compile-database' script. Not now.

Yes, this has all been moved into “location-importer” so that we separate code and data in two different Git repositories.

Best,
-Michael

> 
> -- 
> --gv