Location January 2018

location@lists.ipfire.org

2 participants
5 discussions

A quick status update about the location database
by Peter Müller 24 Jan '18

24 Jan '18

Hello, just a quick status update about the location database ("libloc"). I did some research about how (and where) to gather the data for it the other day. Basically, we just need these information: - IPv4/IPv6 network range - assigned country code - Autonomous System Number (ASN) the network belongs to - description of the network [helpful, but optional in first place] The RIPE provides a documentation about the different files on their FTP server here: https://www.ripe.net/manage-ips-and-asns/db/support/documentation/update-re… The database (either split or en bloc) can be downloaded here: https://ftp.ripe.net/ripe/dbase/ According to the first link, the files ripe.db.inetnum and ripe.db.inet6num contain the allocations of the IP address space. This does not mean that they are actually announced via BGP at the moment, but we agreed not to care about that since a network might be down for maintenance or some other reasons. However, only the network ranges and country codes can be extracted from inet(6)num directly. It seems like we have to look at the route database for anything else (ASN, description), which is located at ripe.db.route. The network range needs to be converted to CIDR first, since the formats differ between inet(6)num and route. But since an AS might announce bigger ranges (say a /15) than the network's size itself (i.e. /29), it might make sense to extract the route announcements in fist place. We already know ASN and some descriptions then. After that, inet(6)num can be parsed to gather networks in that range, and to extract their actual network range and country code. I think it makes sense to keep the description of the AS set here. The last two steps consist in checking if either the complete ASN or a certain network is listed in one of the override files. Finally, the stuff can be dumped into the libloc database. Apart from LACNIC (Latin America and Caribbean), every RIR offers a complete database for download: - ARIN: https://ftp.arin.net/pub/rr/arin.db - AFRINIC: https://ftp.afrinic.net/pub/dbase/afrinic.db.gz - APNIC: https://ftp.apnic.net/apnic/whois/ (split DB only) - RIPE: https://ftp.ripe.net/ripe/dbase/ripe.db.gz LACNIC only provides a list of IP ranges (and decimal encoded subnets - *ugh*) and country codes over here: http://ftp.lacnic.net/pub/stats/lacnic/delegated-lacnic-extended-latest I have no idea at the moment how to enumerate ASN and ASdescr for them... Overall, the RIPE DB looks quite tidy, but we always have a huge number of side effects (one network range is announced by two ASNs, ...) making thinks ugly. I tried to write a Python script for the jobs listed above, but eventually failed since my programming skills are very limited and I needed some layer (perhaps a SQLite DB?) to store partial information. Unfortunately, it seems like that job is much more complicated than I initially thought. Does anybody have an idea about how to store incomplete information here? Let's say, we already know ASN, description and CIDR, but neither all possible networks in that range nor their country... Any help is highly appreciated. Thanks in advance. Best regards, Peter Müller

2 1

[PATCH] add override country code to A1 overrides
by Peter Müller 18 Jan '18

18 Jan '18

Adding a "newcountry" flag for each AS (no IP ranges currently listed here) in the A1 override file. This makes parsing of several override files much easier, since we do not need to care about file names, for example. Signed-off-by: Peter Müller <peter.mueller(a)link38.eu> --- country-overrides/override-a1.txt | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/country-overrides/override-a1.txt b/country-overrides/override-a1.txt index edb91e0..3de3001 100644 --- a/country-overrides/override-a1.txt +++ b/country-overrides/override-a1.txt @@ -19,15 +19,19 @@ net: AS10914 descr: Cypherpunk Privacy Network remarks: VPN provider +newcountry: A1 net: AS37560 descr: Cyberdyne S.A. remarks: VPN and Tor relay provider +newcountry: A1 net: AS396507 descr: Emerald Onion remarks: Tor relay provider +newcountry: A1 net: AS51852 descr: Private Layer INC remarks: VPN provider +newcountry: A1 -- 2.13.6

1 0

[PATCH] A1 overrides: Sort them by ASN
by Peter Müller 16 Jan '18

16 Jan '18

Signed-off-by: Peter Müller <peter.mueller(a)link38.eu> --- country-overrides/override-a1.txt | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/country-overrides/override-a1.txt b/country-overrides/override-a1.txt index a99acf8..edb91e0 100644 --- a/country-overrides/override-a1.txt +++ b/country-overrides/override-a1.txt @@ -16,18 +16,18 @@ # generated by another script. # -net: AS37560 -descr: Cyberdyne S.A. -remarks: VPN and Tor relay provider - net: AS10914 descr: Cypherpunk Privacy Network remarks: VPN provider -net: AS51852 -descr: Private Layer INC -remarks: VPN provider +net: AS37560 +descr: Cyberdyne S.A. +remarks: VPN and Tor relay provider net: AS396507 descr: Emerald Onion remarks: Tor relay provider + +net: AS51852 +descr: Private Layer INC +remarks: VPN provider -- 2.13.6

1 0

[PATCH] country override files: inital checkin
by Peter Müller 07 Jan '18

07 Jan '18

Initially create manual override files for country data. Signed-off-by: Peter Müller <peter.mueller(a)link38.eu> --- country-overrides/override-a1.txt | 18 ++++++++++++++++++ country-overrides/override-a2.txt | 11 +++++++++++ country-overrides/override-a3.txt | 10 ++++++++++ country-overrides/override-other.txt | 12 ++++++++++++ 4 files changed, 51 insertions(+) create mode 100644 country-overrides/override-a1.txt create mode 100644 country-overrides/override-a2.txt create mode 100644 country-overrides/override-a3.txt create mode 100644 country-overrides/override-other.txt diff --git a/country-overrides/override-a1.txt b/country-overrides/override-a1.txt new file mode 100644 index 0000000..08eabfd --- /dev/null +++ b/country-overrides/override-a1.txt @@ -0,0 +1,18 @@ +# +# override-a1 [.txt] +# +# This file contains Autonomous Systems (AS) or IP networks/adresses +# which are - in fact or with a high level of confidence - anonymous +# proxies (special country code: A1). +# +# Since it does not make sense to assign them to a county, they +# will be flagged as "A1" in the database. +# +# Although we do not consider them to be bad entirely, they might +# be unwanted in certain scenarios. +# +# Please note only long-living Tor relay providers with static IPs +# are listed here, as the list of all Tor relays will be dynamically +# generated by another script. +# + diff --git a/country-overrides/override-a2.txt b/country-overrides/override-a2.txt new file mode 100644 index 0000000..2ce00d0 --- /dev/null +++ b/country-overrides/override-a2.txt @@ -0,0 +1,11 @@ +# +# override-a2 [.txt] +# +# This file contains Autonomous Systems (AS) or IP networks/addresses +# which are - in fact or with a high level of confidence - belonging +# to satellite network providers (special country code: A2). +# +# Since a satellite uplink connection is possible from almost +# anywhere in the world, it does not make sense to assign them to a +# specific country. They will be flagged as "A2" in the database. +# diff --git a/country-overrides/override-a3.txt b/country-overrides/override-a3.txt new file mode 100644 index 0000000..4fbcdeb --- /dev/null +++ b/country-overrides/override-a3.txt @@ -0,0 +1,10 @@ +# +# override-a3 [.txt] +# +# This file contains Autonomous Systems (AS) or IP networks/addresses +# which are - in fact or with a high level of confidence - believed +# to be worldwide anycast instances (special country codes: A3). +# +# It does not make sense to assign them to a certain country, they +# will be flagged as "A3" in the database. +# diff --git a/country-overrides/override-other.txt b/country-overrides/override-other.txt new file mode 100644 index 0000000..d7ba5b9 --- /dev/null +++ b/country-overrides/override-other.txt @@ -0,0 +1,12 @@ +# +# override-other [.txt] +# +# This file contains Autonomous Systems (AS) or IP networks/addresses +# whose country information in the WHOIS is not identical to their +# physical location and which do not match to one of the special categories +# A[1-3]. +# +# Such networks might be legitimate (poorly maintained WHOIS data) +# or hostile (faked WHOIS data in order to bypass country filtering or +# to hide the owners' identity). +# -- 2.13.6

2 3

[PATCH] A1 override: add AS396507 (Emerald Onion)
by Peter Müller 07 Jan '18

07 Jan '18

Signed-off-by: Peter Müller <peter.mueller(a)link38.eu> --- country-overrides/override-a1.txt | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/country-overrides/override-a1.txt b/country-overrides/override-a1.txt index 0936994..a99acf8 100644 --- a/country-overrides/override-a1.txt +++ b/country-overrides/override-a1.txt @@ -27,3 +27,7 @@ remarks: VPN provider net: AS51852 descr: Private Layer INC remarks: VPN provider + +net: AS396507 +descr: Emerald Onion +remarks: Tor relay provider -- 2.13.6

2 2

2024

2023

2022

2021

2020

2019

2018

2017

Location January 2018