libloc 0.9.18 has been released

libloc 0.9.18 has been released

[Michael Tremer]

Hello everyone,

This is the official announcement for the most recent release of libloc: libloc 0.9.18. Is comes with the following changes:

* A new Lua module has been added as announced here: https://www.ipfire.org/blog/ipfire-location-lua-bindings-for-fun-and-profit

* The algorithm to detect bogons and duplicates in the tree have massively been improved and should be nearing their theoretical maximum in terms of performance. 

* A large number of stability and correctness fixes have been rolled out. These mostly affect the code generating the database.

* We now have a small Jenkins pipeline which will check if the library still builds for a couple of major Linux distributions and various architectures.

Python:

* AS and Country objects are now hashable and support rich comparison operations

Importer:

* Exporting the database is around 200x faster due to eliminating any excessive joins. Instead a new temporary table will be created and a temporary index will be used to apply various updates to the data from various sources inside the database. That allows us to create the export iteratively instead of having one large query that runs for forever. An export that formerly took around 17-20 hrs(!) will now take only ~5 mins.

* A new source for human-friendly names for Autonomous Systems registered with ARIN has been added

* Importing feeds from AWS and Spamhaus has been split off into separate database tables. This will allow us to import them separately and prioritise our own rewrites over them.

* The ARIN parser has been refactored based on csv.DictReader(), parsers for the AWS and Spamhaus feeds have been rewritten, too

* Geofeeds are now fetched concurrently with a unified downloader

* Certain country codes will be entirely ignored. Currently this is YU for former Yugoslavia and ZZ which is used to say “no country”

* Country codes can now be corrected on the fly. This is used to change UK to GB as only the latter is the valid country code for the United Kingdom.

* Countries that are not on our list will not be imported any more.

* Networks larger than /4 for IPv6, and /10 for IPv4 won’t be imported any more. This avoids that we propagate any issues in the global routing table into the database.

Thanks to everyone who has contributed to this release.

This release has *a lot* of large changes and we would appreciate any feedback or bug reports in case there are any regressions. Also, please help us to keep improving this software and publishing our database by donating: https://www.ipfire.org/donate.

If you are a packager for any of the Linux distributions that already ship libloc, please update to this release. If you are a packager for a distribution that does not have libloc, yet, please add it to your distribution.

You can find the source here:

https://git.ipfire.org/?p=location/libloc.git;a=shortlog;h=refs/tags/0.9.18
https://source.ipfire.org/releases/libloc/libloc-0.9.18.tar.gz SHA256 16c76689fa0ea8ece4fca6abcf942f236c48440387bee1d9331ea937ccd7b87d

Best,
-Michael