September 2020 - Location - lists.ipfire.org

[PATCH v2 1/3] location-importer.in: avoid violating NOT NULL constraints during JOIN
by Peter Müller 24 Sep '20

24 Sep '20

Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org> --- src/python/location-importer.in | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/python/location-importer.in b/src/python/location-importer.in index f5ae4a9..77952f2 100644 --- a/src/python/location-importer.in +++ b/src/python/location-importer.in @@ -360,7 +360,7 @@ class CLI(object): ON COMMIT DROP; CREATE UNIQUE INDEX _autnums_number ON _autnums(number); - CREATE TEMPORARY TABLE _organizations(handle text, name text) + CREATE TEMPORARY TABLE _organizations(handle text, name text NOT NULL) ON COMMIT DROP; CREATE UNIQUE INDEX _organizations_handle ON _organizations(handle); """) @@ -373,7 +373,7 @@ class CLI(object): self.db.execute(""" INSERT INTO autnums(number, name) SELECT _autnums.number, _organizations.name FROM _autnums - LEFT JOIN _organizations ON _autnums.organization = _organizations.handle + JOIN _organizations ON _autnums.organization = _organizations.handle ON CONFLICT (number) DO UPDATE SET name = excluded.name; """) -- 2.26.2

2 3

[PATCH] overrides/override-{a1,other}.txt: weekly batch of override updates
by Peter Müller 19 Sep '20

19 Sep '20

Overrides have been handed in as more or less atomic changes with separate patches or patchsets. Given the volatility of those, this approach does not seem to be worth the effort any more, which is why this patch introduces weekly (?) batching of those changes in one big patch. The contents of overrides have been researched with caution, sometimes even exhaustively, which is why they are considered to be safe to be applied en bloc. Cc: Michael Tremer <michael.tremer(a)ipfire.org> Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org> --- overrides/override-a1.txt | 50 +++++++++++++++++++----------------- overrides/override-other.txt | 44 +++++++++++++++++++++++++++++-- 2 files changed, 68 insertions(+), 26 deletions(-) diff --git a/overrides/override-a1.txt b/overrides/override-a1.txt index 82d52ff..9f64dd8 100644 --- a/overrides/override-a1.txt +++ b/overrides/override-a1.txt @@ -28,11 +28,6 @@ descr: Maginfo remarks: VPN provider is-anonymous-proxy: yes -aut-num: AS10914 -descr: Cypherpunk Privacy Network -remarks: VPN provider -is-anonymous-proxy: yes - aut-num: AS23762 descr: VPNsolutions Pty Ltd remarks: VPN provider @@ -43,6 +38,12 @@ descr: VPN de Mexico, S.A. de C.V. remarks: VPN provider is-anonymous-proxy: yes +aut-num: AS35029 +descr: WebLine LTD +remarks: (Rogue) VPN provider +is-anonymous-proxy: yes +country: RU + aut-num: AS37560 descr: Cyberdyne S.A. remarks: VPN and Tor relay provider located in SE @@ -64,6 +65,11 @@ descr: BeeVPN ApS remarks: VPN provider is-anonymous-proxy: yes +aut-num: AS51446 +descr: SP Argaev Artem Sergeyevich / Foundation Respect My Privacy +remarks: VPN provider [high confidence, but not proofed] +is-anonymous-proxy: yes + aut-num: AS51852 descr: Private Layer INC remarks: VPN provider (claims PA or BZ for some prefixes, but they are all hosted in CH) @@ -100,11 +106,6 @@ descr: Anonymizer, Inc. remarks: VPN provider is-anonymous-proxy: yes -aut-num: AS206106 -descr: Privacy Ukraine LLC -remarks: VPN provider -is-anonymous-proxy: yes - aut-num: AS208294 descr: Markus Koch remarks: Tor relay provider @@ -119,6 +120,7 @@ aut-num: AS208476 descr: Danilenko, Artyom remarks: (Rogue) VPN provider is-anonymous-proxy: yes +country: EU aut-num: AS209623 descr: David Craig @@ -551,6 +553,11 @@ descr: TopNet ISP VPN remarks: VPN provider is-anonymous-proxy: yes +net: 88.214.27.0/24 +descr: ThinkTech Technology Industrial CO. Limited +remarks: VPN provider +is-anonymous-proxy: yes + net: 94.199.160.0/23 descr: MIK Telecom VPN pool remarks: VPN provider @@ -956,8 +963,8 @@ descr: VKVPN remarks: VPN provider is-anonymous-proxy: yes -net: 185.244.30.0/24 -descr: Freedom of Speech VPN / nVPN / David Craig +net: 185.244.28.0/22 +descr: Freedom of Speech VPN / nVPN / David Craig / ... remarks: (Rogue) VPN provider is-anonymous-proxy: yes @@ -1086,18 +1093,8 @@ descr: Anonymouse / Anonymous S.A. remarks: VPN and open proxy provider is-anonymous-proxy: yes -net: 194.5.97.0/24 -descr: Privacy Online / Danilenko, Artyom -remarks: (Rogue) VPN provider -is-anonymous-proxy: yes - -net: 194.5.98.0/24 -descr: Privacy Online / Danilenko, Artyom -remarks: (Rogue) VPN provider -is-anonymous-proxy: yes - -net: 194.5.99.0/24 -descr: Privacy Online / Danilenko, Artyom +net: 194.5.96.0/22 +descr: Privacy Online / Danilenko, Artyom / ... remarks: (Rogue) VPN provider is-anonymous-proxy: yes @@ -1191,6 +1188,11 @@ descr: DSTCom NAT Pool (VPN and RTSP) remarks: VPN provider is-anonymous-proxy: yes +net: 203.159.81.0/24 +descr: VPN Consumer Network Services +remarks: VPN provider +is-anonymous-proxy: yes + net: 205.185.193.0/24 descr: SecuredConnectivity remarks: VPN provider diff --git a/overrides/override-other.txt b/overrides/override-other.txt index 8d2cf03..f8e6799 100644 --- a/overrides/override-other.txt +++ b/overrides/override-other.txt @@ -23,6 +23,11 @@ descr: IP Interactive UG (haftungsbeschraenkt) remarks: ISP located in BG, but RIR data for announced prefixes contain garbage country: BG +aut-num: AS40034 +descr: Confluence Networks Inc. +remarks: fake offshore location (VG), traces back to Austin, TX, US +country: US + aut-num: AS42397 descr: Bunea TELECOM SRL remarks: ISP located in RO, but some RIR data for announced prefixes contain garbage @@ -43,11 +48,21 @@ descr: FiberXpress BV remarks: bulletproof ISP (related to AS202425) located in NL country: NL +aut-num: AS58271 +descr: FOP Gubina Lubov Petrivna +remarks: bulletproof ISP operating from a war zone in eastern UA +country: UA + aut-num: AS62355 descr: Network Dedicated SAS remarks: bulletproof ISP, claims to be located in CH, but traces to NL country: NL +aut-num: AS134548 +descr: DXTL Tseung Kwan O Service +remarks: tampers with RIR data, traces back to AP region +country: AP + aut-num: AS201133 descr: Verdina Ltd. remarks: ISP located in BG, but RIR data for announced prefixes contain garbage @@ -70,14 +85,24 @@ country: NL aut-num: AS207616 descr: Altrosky Technology Ltd. -remarks: fake offshore location (SC), traces back to CZ -country: CZ +remarks: fake offshore location (SC), traces back to CZ and NL +country: EU aut-num: AS209132 descr: Alviva Holding Limited remarks: ISP located in BG, but RIR data for announced prefixes contain garbage country: BG +aut-num: AS209272 +descr: Alviva Holding Limited +remarks: bulletproof ISP operating from a war zone in eastern UA +country: UA + +aut-num: AS213058 +descr: Private Internet Hosting LTD +remarks: bulletproof ISP located in RU +country: RU + net: 31.220.0.0/22 descr: Amarutu Technology Ltd. / KoDDoS / ESecurity remarks: fake offshore location (BZ), traces back to NL @@ -93,6 +118,11 @@ descr: FlokiNET Ltd. remarks: fake offshore location (SC), traces back to RO country: RO +net: 45.145.36.0/22 +descr: GlobalCache Technology CO., Ltd. +remarks: claims to be located in DE, but traces back to HK +country: HK + net: 47.60.0.0/14 descr: Vodafone US Inc. remarks: large Vodafone IP chunk used in ES, but assigned by ARIN (inaccurate data) @@ -147,3 +177,13 @@ net: 2a02:e00:ffe7::/48 descr: Bradler & Krantz GmbH & Co. KG remarks: allocated to DE, but actually announced from LT country: LT + +net: 2a02:e00:ffe8::/48 +descr: Bradler & Krantz GmbH & Co. KG +remarks: allocated to DE, but actually announced from MD +country: MD + +net: 2a02:e00:ffe9::/48 +descr: Bradler & Krantz GmbH & Co. KG +remarks: allocated to DE, but actually announced from GB +country: GB -- 2.26.2

1 0

[PATCH] location: Fix message sting if the database has been recently updated
by Stefan Schantl 16 Sep '20

16 Sep '20

Signed-off-by: Stefan Schantl <stefan.schantl(a)ipfire.org> --- src/python/location.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/python/location.in b/src/python/location.in index 8cdd140..3ba209d 100644 --- a/src/python/location.in +++ b/src/python/location.in @@ -413,7 +413,7 @@ class CLI(object): # Check if the database has recently been updated if t >= (now - delta): log.info( - _("The datase has recently be updated recently (%s)") % \ + _("The database has been updated recently(%s)") % \ format_timedelta(now - t), ) return 3 -- 2.20.1

2 1

[PATCH 1/4] perl: Add get_country_name() function.
by Stefan Schantl 10 Sep '20

10 Sep '20

This function can be used to get the stored name for a given country code. Signed-off-by: Stefan Schantl <stefan.schantl(a)ipfire.org> --- src/perl/Location.xs | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/src/perl/Location.xs b/src/perl/Location.xs index 1cb2e21..dcf3f0d 100644 --- a/src/perl/Location.xs +++ b/src/perl/Location.xs @@ -243,6 +243,28 @@ lookup_asn(db, address) # # Get functions # +SV* +get_country_name(db, ccode) + struct loc_database* db; + char* ccode; + + CODE: + RETVAL = &PL_sv_undef; + + // Lookup country code + struct loc_country *country; + int err = loc_database_get_country(db, &country, ccode); + if(!err) { + // Extract the name for the given country code. + const char* country_name = loc_country_get_name(country); + RETVAL = newSVpv(country_name, strlen(country_name)); + + loc_country_unref(country); + } + + OUTPUT: + RETVAL + SV* get_continent_code(db, ccode) struct loc_database* db; -- 2.20.1

1 3

[PATCH 1/3] country-overrides: add 47.60.0.0/14
by Peter Müller 08 Sep '20

08 Sep '20

Triggered by: https://community.ipfire.org/t/location-filter-since-core-148/3149/13 Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org> --- overrides/override-other.txt | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/overrides/override-other.txt b/overrides/override-other.txt index 6035962..5e06a6d 100644 --- a/overrides/override-other.txt +++ b/overrides/override-other.txt @@ -63,6 +63,11 @@ descr: FlokiNET Ltd. remarks: fake offshore location (SC), traces back to RO country: RO +net: 47.60.0.0/14 +descr: Vodafone US Inc. +remarks: large Vodafone IP chunk used in ES, but assigned by ARIN (inaccurate data) +country: ES + net: 85.202.80.0/24 descr: Amarutu Technology Ltd. / KoDDoS / ESecurity remarks: fake offshore location (BZ), traces back to US -- 2.26.2

1 2

[PATCH 1/3] location-importer.in: avoid violating NOT NULL constraints during JOIN
by Peter Müller 06 Sep '20

06 Sep '20

Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org> --- src/python/location-importer.in | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/python/location-importer.in b/src/python/location-importer.in index f5ae4a9..77952f2 100644 --- a/src/python/location-importer.in +++ b/src/python/location-importer.in @@ -360,7 +360,7 @@ class CLI(object): ON COMMIT DROP; CREATE UNIQUE INDEX _autnums_number ON _autnums(number); - CREATE TEMPORARY TABLE _organizations(handle text, name text) + CREATE TEMPORARY TABLE _organizations(handle text, name text NOT NULL) ON COMMIT DROP; CREATE UNIQUE INDEX _organizations_handle ON _organizations(handle); """) @@ -373,7 +373,7 @@ class CLI(object): self.db.execute(""" INSERT INTO autnums(number, name) SELECT _autnums.number, _organizations.name FROM _autnums - LEFT JOIN _organizations ON _autnums.organization = _organizations.handle + JOIN _organizations ON _autnums.organization = _organizations.handle ON CONFLICT (number) DO UPDATE SET name = excluded.name; """) -- 2.20.1

1 2

[PATCH] override-others.txt: fix RIR garbage for AS24700
by Peter Müller 04 Sep '20

04 Sep '20

Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org> --- overrides/override-other.txt | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/overrides/override-other.txt b/overrides/override-other.txt index 6035962..cb02a5b 100644 --- a/overrides/override-other.txt +++ b/overrides/override-other.txt @@ -13,6 +13,11 @@ # Please keep this file sorted. # +aut-num: AS24700 +descr: Yes Networks Unlimited Ltd +remarks: traces to UA, but some RIR entries seem to contain garbage (VG) +country: UA + aut-num: AS48090 descr: PPTECHNOLOGY LIMITED remarks: bulletproof ISP (related to AS204655) located in NL -- 2.26.2

1 0

Re-introducing inetnum parser, first attempt
by Peter Müller 01 Sep '20

01 Sep '20

Good evening Michael, below comes the diff of a rather hacky attempt to bring back the intetnum parser we've had in the past. Since I am not quite sure about that INSERT INTO networks() SQL statement and handling conflicts with extended source files there, I thought letting you have a look at it might be a good idea. :-) What do you think? Is this the right direction? Thanks, and best regards, Peter Müller diff --git a/src/python/importer.py b/src/python/importer.py index de20f37..586bd97 100644 --- a/src/python/importer.py +++ b/src/python/importer.py @@ -30,8 +30,8 @@ WHOIS_SOURCES = ( "https://ftp.afrinic.net/pub/pub/dbase/afrinic.db.gz", # Asia Pacific Network Information Centre - #"https://ftp.apnic.net/apnic/whois/apnic.db.inet6num.gz", - #"https://ftp.apnic.net/apnic/whois/apnic.db.inetnum.gz", + "https://ftp.apnic.net/apnic/whois/apnic.db.inet6num.gz", + "https://ftp.apnic.net/apnic/whois/apnic.db.inetnum.gz", #"https://ftp.apnic.net/apnic/whois/apnic.db.route6.gz", #"https://ftp.apnic.net/apnic/whois/apnic.db.route.gz", "https://ftp.apnic.net/apnic/whois/apnic.db.aut-num.gz", @@ -45,8 +45,8 @@ WHOIS_SOURCES = ( # XXX ??? # Réseaux IP Européens - #"https://ftp.ripe.net/ripe/dbase/split/ripe.db.inet6num.gz", - #"https://ftp.ripe.net/ripe/dbase/split/ripe.db.inetnum.gz", + "https://ftp.ripe.net/ripe/dbase/split/ripe.db.inet6num.gz", + "https://ftp.ripe.net/ripe/dbase/split/ripe.db.inetnum.gz", #"https://ftp.ripe.net/ripe/dbase/split/ripe.db.route6.gz", #"https://ftp.ripe.net/ripe/dbase/split/ripe.db.route.gz", "https://ftp.ripe.net/ripe/dbase/split/ripe.db.aut-num.gz", diff --git a/src/python/location-importer.in b/src/python/location-importer.in index f5ae4a9..4d7cec4 100644 --- a/src/python/location-importer.in +++ b/src/python/location-importer.in @@ -393,6 +393,10 @@ class CLI(object): if line.startswith("aut-num:"): return self._parse_autnum_block(block) + # inetnum + if line.startswith("inet6num:") or line.startswith("inetnum:"): + return self._parse_inetnum_block(block) + # organisation elif line.startswith("organisation:"): return self._parse_org_block(block) @@ -422,6 +426,78 @@ class CLI(object): autnum.get("asn"), autnum.get("org"), ) + def _parse_inetnum_block(self, block): + logging.debug("Parsing inetnum block:") + + inetnum = {} + for line in block: + logging.debug(line) + + # Split line + key, val = split_line(line) + + if key == "inetnum": + start_address, delim, end_address = val.partition("-") + + # Strip any excess space + start_address, end_address = start_address.rstrip(), end_address.strip() + + # Skip invalid blocks + if start_address in ["0.0.0.0", "::/0", "0::/0",]: + return + + # Convert to IP address + try: + start_address = ipaddress.ip_address(start_address) + end_address = ipaddress.ip_address(end_address) + except ValueError: + logging.warning("Could not parse line: %s" % line) + return + + # Set prefix to default + prefix = 32 + + # Count number of addresses in this subnet + num_addresses = int(end_address) - int(start_address) + if num_addresses: + prefix -= math.log(num_addresses, 2) + + inetnum["inetnum"] = "%s/%.0f" % (start_address, prefix) + + elif key == "inet6num": + # Skip invalid blocks + if val in ["0.0.0.0", "::/0", "0::/0",]: + return + + inetnum[key] = val + + elif key == "netname": + inetnum[key] = val + + elif key == "country": + if val == "UNITED STATES": + val = "US" + + inetnum[key] = val.upper() + + elif key == "descr": + if key in inetnum: + inetnum[key] += "\n%s" % val + else: + inetnum[key] = val + + # Skip empty objects + if not inetnum: + return + + network = ipaddress.ip_network(inetnum.get("inet6num") or inetnum.get("inetnum"), strict=False) + + self.db.execute("INSERT INTO networks(network, country) \ + VALUES(%s, %s) ON CONFLICT (network) DO \ + UPDATE SET country = excluded.country", + "%s" % str(network), inetnum.get("country"), + ) + def _parse_org_block(self, block): org = {} for line in block:

1 0

[PATCH] A2 overrides: add various missing satellite internet providers
by Peter Müller 01 Sep '20

01 Sep '20

Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org> --- overrides/override-a2.txt | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/overrides/override-a2.txt b/overrides/override-a2.txt index b0109f7..88783d8 100644 --- a/overrides/override-a2.txt +++ b/overrides/override-a2.txt @@ -132,6 +132,16 @@ descr: Eutelsat S.A. remarks: Satellite Internet provider is-satellite-provider: yes +aut-num: AS35550 +descr: SatTel Corporation, Ltd. +remarks: Satellite Internet provider +is-satellite-provider: yes + +aut-num: AS39102 +descr: SatTel Corporation, Ltd. +remarks: Satellite Internet provider +is-satellite-provider: yes + aut-num: AS39151 descr: AXESS Networks Solutions Germany GmbH / CeTEL GmbH / GeoLink remarks: Satellite Internet provider @@ -147,6 +157,11 @@ descr: Telenor Satellite AS remarks: Satellite Internet provider is-satellite-provider: yes +aut-num: AS44720 +descr: SatTel Corporation, Ltd. +remarks: Satellite Internet provider +is-satellite-provider: yes + aut-num: AS45232 descr: Spacenet Internet Services Pvt. Ltd. remarks: Satellite Internet provider @@ -202,6 +217,16 @@ descr: IPStar remarks: Satellite Internet provider [high confidence, but not proofed] is-satellite-provider: yes +aut-num: AS139034 +descr: Sattelite Plus Incorporated +remarks: Satellite Internet provider +is-satellite-provider: yes + +aut-num: AS139647 +descr: Alpha Sattelite Network +remarks: Satellite Internet provider +is-satellite-provider: yes + aut-num: AS393582 descr: VIDEO DIRECT SATELLITE & ENTERTAINMENT / WisperISP remarks: Satellite Internet provider -- 2.26.2

1 0

2024

2023

2022

2021

2020

2019

2018

2017

Location September 2020