Question regarding delta between "delegated-ripencc-extended-latest" and Whois output

Question regarding delta between "delegated-ripencc-extended-latest" and Whois output

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 1202 bytes --]

Dear RIPE DB mailing list people,

we, the IPFire development team, just bumped across an IP address somewhere
in the 51.15.0.0/17 network (no abuse, we were just curious :-) ).

While the RIPE Whois server returned "NL" as the country of that subnet [1],
the "delegated-ripencc-extended-latest" file available at ftp.ripe.net [2] only
contains 51.15.0.0/16, which points to "FR".

Since the first one result contains the correct country code of that IP network,
I would like to ask why "delegated-ripencc-extended-latest" does not include
subnets like this, and where the Whois server fetches his answer from.

Did we miss something? Is there another or a better way of fetching precise
subnet data from RIPE? It would be great if someone could point us into the
right direction. :-)

Thanks, and best regards,
Peter Müller

[1] Relevant snippet of the Whois server output:
>     inetnum:         51.15.0.0 - 51.15.63.255
>     [...]
>     country:         NL
>     [...]

[2] https://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-extended-latest,
relevant snippet of this file:
> ripencc|FR|ipv4|51.15.0.0|65536|19930901|assigned|925547da-35fb-4346-b464-aadff095c6ea

Re: [db-wg] Question regarding delta between "delegated-ripencc-extended-latest" and Whois output

[Nick Hilliard]

[-- Attachment #1: Type: text/plain, Size: 1760 bytes --]

Peter Müller via db-wg wrote on 29/06/2020 18:30:
> While the RIPE Whois server returned "NL" as the country of that
> subnet [1], the "delegated-ripencc-extended-latest" file available at
> ftp.ripe.net [2] only contains 51.15.0.0/16, which points to "FR".
> 
> Since the first one result contains the correct country code of that
> IP network, I would like to ask why
> "delegated-ripencc-extended-latest" does not include subnets like
> this, and where the Whois server fetches his answer from.

The entirety of this /16 is assigned to a single organisation 
(Online.net), which is why there's a /16 in 
delegated-ripencc-extended-latest. This file contains only supernets and 
nothing more granular.

Online.net created an inetnum object for 51.15.0.0/17 which sets the 
country to be NL.  This is why the RIPE DB returns NL for that 
particular block.

Incidentally, IP address allocation queries in the RIPE Database now 
return the country where the holder of the block is legally resident, 
not where the IP address is actually used.  The situation with 
51.15.0.0/17 and 51.15.0.0/16 is only the tip of a gigantic iceberg of 
quirks and anomalies, not just because you have inconsistent statements 
about the country, but also because this is legacy (i.e. pre-RIPE) 
address space and there can be multiple inetnum entries in the ripedb 
which may be fully inconsistent with each other and where all, some or 
none of the inetnum records may be an accurate reflection of reality.

I.e. if you're planning to use either the RIPE DB or any other whois db 
for the ipfire geoip blocking mechanism, you may want to consider 
putting up very large and obtrusive warnings about how thoroughly 
inaccurate geoblocking may end up being in practice.

Nick

Re: [db-wg] Question regarding delta between "delegated-ripencc-extended-latest" and Whois output

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 3549 bytes --]

Hello Nick, hello Leo,

thanks for your fast and informative replies. I take the liberty to reply to
both of you at the same time.

> The RIPE Database, queried by Whois or RDAP, serves a different
> purpose from the stats file. The former enables communication between
> operators and supports the routing registry. The latter is to provide
> high level statistical data to researchers.

I see, it did not became clear to us that those file mainly exists for
statistical purposes - we were glad to find something that has a unique
syntax across all RIRs, instead of reintroducing the wheel for each one.

Oh well... :-)

> Incidentally, IP address allocation queries in the RIPE Database now
> return the country where the holder of the block is legally resident,
> not where the IP address is actually used.

Yes. This seems to be a common source of misunderstandings when trying to
assign a country code to an IP network, as most people/applications/databases
do not specify if they are trying to show the jurisdiction of that IP
network, or the country where its machines are physically located. In
my humble opinion, the latter is hard - if any - to determine.

> The situation with 51.15.0.0/17 and 51.15.0.0/16 is only the tip of a
> gigantic iceberg of quirks and anomalies, not just because you have
> inconsistent statements about the country, but also because this is
> legacy (i.e. pre-RIPE) address space and there can be multiple inetnum
> entries in the ripedb which may be fully inconsistent with each other and
> where all, some or none of the inetnum records may be an accurate reflection
> of reality.

Although we have already sensed something in that direction, it is a
bit frustrating to receive that message from someone in the RIPE DB working
group...

What would you do if you need to build a database to determine the
country assigned to an IP address - let's take the one assigned to it by
it's owner, regardless of jurisdiction or physical location -? Would you
collect the most specific inetnums first, and proceed with the less specific
ones?

(Basically, this is what we trying to do as we need to replace the GeoIP
database provided by MaxMind due to license changes - some other open-source
projects are in need of an alternative as well -, and instead of just using
the next commercial provider, we are trying to build an open-source solution.

In case of further interest, please refer to:
https://blog.ipfire.org/post/on-retiring-the-maxmind-geoip-database)

> I.e. if you're planning to use either the RIPE DB or any other whois db
> for the ipfire geoip blocking mechanism, you may want to consider putting
> up very large and obtrusive warnings about how thoroughly inaccurate
> geoblocking may end up being in practice.

I believe we are more or less aware of that inaccuracies. Personally, I consider
firewall rules based on Autonomous Systems Numbers to be _way_ more precise
and practical - countries like US or NL are hard to block in productive environments.

We are keen to raise awareness of those issues in our user community as
well, but unfortunately are unable to drop the ability of selecting countries
as source or destination of a firewall rule completely. For a quick look, it
also might be interesting to show some geographical information so very
unusual connections are easier to identify.

Anyway: Thanks for your replies so far and thanks in advance for the upcoming ones. :-)

Thanks, and best regards,
Peter Müller

Re: [db-wg] Question regarding delta between "delegated-ripencc-extended-latest" and Whois output

[Nick Hilliard]

[-- Attachment #1: Type: text/plain, Size: 708 bytes --]

Peter Müller wrote on 29/06/2020 20:54:
> What would you do if you need to build a database to determine the
> country assigned to an IP address - let's take the one assigned to it by
> it's owner, regardless of jurisdiction or physical location -?

There's no short or easy answer to this - if there were, there would be 
lots of geo-ip databases available on the internet.

You would need to distil the information from a lot of separate sources 
(e.g. whois databases, detailed dfz routing dumps from multiple sources, 
pings from beacons to test rtt, possibly dns as a fuzzy input, etc).  It 
would be a highly labour intensive process and you would be left with 
large numbers of question marks.

Nick

Re: [db-wg] Question regarding delta between "delegated-ripencc-extended-latest" and Whois output

[Leo Vegoda]

[-- Attachment #1: Type: text/plain, Size: 760 bytes --]

On Mon, Jun 29, 2020 at 1:22 PM Nick Hilliard via db-wg <db-wg(a)ripe.net> wrote:
> Peter Müller wrote on 29/06/2020 20:54:
> > What would you do if you need to build a database to determine the
> > country assigned to an IP address - let's take the one assigned to it by
> > it's owner, regardless of jurisdiction or physical location -?
>
> There's no short or easy answer to this - if there were, there would be
> lots of geo-ip databases available on the internet.

Also, GeoIP data is collected to do different things. Some people care
about jurisdiction, other care about territorial rights for media
distribution, other care about language, and others care about
latency. No single database is going to be good at doing all those
things.