Hello,
just a quick status update about the location database ("libloc"). I did some research about how (and where) to gather the data for it the other day.
Basically, we just need these information: - IPv4/IPv6 network range - assigned country code - Autonomous System Number (ASN) the network belongs to - description of the network [helpful, but optional in first place]
The RIPE provides a documentation about the different files on their FTP server here: https://www.ripe.net/manage-ips-and-asns/db/support/documentation/update-ref... The database (either split or en bloc) can be downloaded here: https://ftp.ripe.net/ripe/dbase/
According to the first link, the files ripe.db.inetnum and ripe.db.inet6num contain the allocations of the IP address space. This does not mean that they are actually announced via BGP at the moment, but we agreed not to care about that since a network might be down for maintenance or some other reasons.
However, only the network ranges and country codes can be extracted from inet(6)num directly. It seems like we have to look at the route database for anything else (ASN, description), which is located at ripe.db.route.
The network range needs to be converted to CIDR first, since the formats differ between inet(6)num and route. But since an AS might announce bigger ranges (say a /15) than the network's size itself (i.e. /29), it might make sense to extract the route announcements in fist place. We already know ASN and some descriptions then.
After that, inet(6)num can be parsed to gather networks in that range, and to extract their actual network range and country code. I think it makes sense to keep the description of the AS set here.
The last two steps consist in checking if either the complete ASN or a certain network is listed in one of the override files. Finally, the stuff can be dumped into the libloc database.
Apart from LACNIC (Latin America and Caribbean), every RIR offers a complete database for download: - ARIN: https://ftp.arin.net/pub/rr/arin.db - AFRINIC: https://ftp.afrinic.net/pub/dbase/afrinic.db.gz - APNIC: https://ftp.apnic.net/apnic/whois/ (split DB only) - RIPE: https://ftp.ripe.net/ripe/dbase/ripe.db.gz
LACNIC only provides a list of IP ranges (and decimal encoded subnets - *ugh*) and country codes over here: http://ftp.lacnic.net/pub/stats/lacnic/delegated-lacnic-extended-latest I have no idea at the moment how to enumerate ASN and ASdescr for them...
Overall, the RIPE DB looks quite tidy, but we always have a huge number of side effects (one network range is announced by two ASNs, ...) making thinks ugly.
I tried to write a Python script for the jobs listed above, but eventually failed since my programming skills are very limited and I needed some layer (perhaps a SQLite DB?) to store partial information. Unfortunately, it seems like that job is much more complicated than I initially thought.
Does anybody have an idea about how to store incomplete information here? Let's say, we already know ASN, description and CIDR, but neither all possible networks in that range nor their country...
Any help is highly appreciated. Thanks in advance.
Best regards, Peter Müller