Hello,

How about we add the country to the list and mark the continent with a “-“:

AF AS Afghanistan YU - Yugoslavia

-Michael

On 10 Apr 2021, at 13:32, Peter Müller peter.mueller@ipfire.org wrote:

Hello Michael,

thanks for your reply.

Yes, having this configurable in countries.txt would be nice indeed. Do you propose a certain syntax for this?

And yes, a tuple is a better idea here. I will wait for your reply and submit a second version of this patch then.

Thanks, and best regards, Peter Müller

...

Hello,

• would we not want this to be configurable in countries.txt?

• The list should probably be a tuple.

-Michael

...

On 1 Apr 2021, at 20:57, Peter Müller peter.mueller@ipfire.org wrote:

Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the time of writing. Network owners/operators putting these countries into their RIR data objects are either completely braindead or doing so for hostile reasons.

While we might correct these locations to something useful by manually creating overrides for them, the rationale behind this patch is not to let these countries appear on productive systems in the first place, as we know they _cannot_ be true.

Therefore, this patch skips any network object that has either AQ or BV country code set.

See also: https://lists.ipfire.org/pipermail/location/2020-October/000199.html

Signed-off-by: Peter Müller peter.mueller@ipfire.org

src/python/location-importer.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/python/location-importer.in b/src/python/location-importer.in index 1e08458..ac678dc 100644 --- a/src/python/location-importer.in +++ b/src/python/location-importer.in @@ -624,7 +624,7 @@ class CLI(object): return

# Skip objects with unknown country codes

• if validcountries and inetnum.get("country") not in validcountries:
  

• if validcountries and (inetnum.get("country") not in validcountries or inetnum.get("country") in ["AQ", "BV"]):
  log.warning("Skipping network with bogus country '%s': %s" % \
  	(inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum")))
  return
  

-- 2.26.2

Hi,

On 12 Apr 2021, at 18:26, Peter Müller peter.mueller@ipfire.org wrote:

Hello Michael,

thanks for your reply.

I believe you are mixing up two distinct scenarios here:

I am trying not to. I just added the name because I struggle to remember some country codes.

(a) Historic ISO-3166-X abbreviations for countries no longer existing anymore, such as YU.

They technically do.

They are still reserved in the ISO 3166 database and they show up (in this case) in the RIPE database. So we will have to handle them no matter what.

YU does not have a succeeding national body (like Eastern Germany would have the Federal Republic of Germany) so we cannot automatically rewrite them.

(b) Country codes for existing nations, where we believe with a high level of confidence no IP networks are located in any way, either physically or legally. AQ and BV are the most notable examples known to me at the time of writing.

Since it does not make sense to add historic country codes, I would suggest to leave networks assigned to (a) countries as they are, manually creating overrides for them, as I did before.

For (b), simply scrubbing out the country or region they belong to seems to be too harsh to me, as we use this information for several things, and should not abuse it for our own purposes.

(I assume) we still have AS information about those networks and we should keep at least that.

Therefore, I rather suggest adding an additional column to countries.txt for indicating whether this country code is acceptable to us or not.

Okay. What would you suggest?

-Michael

Thanks, and best regards, Peter Müller

...

Hello,

How about we add the country to the list and mark the continent with a “-“:

AF AS Afghanistan YU - Yugoslavia

-Michael

...

On 10 Apr 2021, at 13:32, Peter Müller peter.mueller@ipfire.org wrote:

Hello Michael,

thanks for your reply.

Yes, having this configurable in countries.txt would be nice indeed. Do you propose a certain syntax for this?

And yes, a tuple is a better idea here. I will wait for your reply and submit a second version of this patch then.

Thanks, and best regards, Peter Müller

...

Hello,

• would we not want this to be configurable in countries.txt?

• The list should probably be a tuple.

-Michael

...

On 1 Apr 2021, at 20:57, Peter Müller peter.mueller@ipfire.org wrote:

Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the time of writing. Network owners/operators putting these countries into their RIR data objects are either completely braindead or doing so for hostile reasons.

While we might correct these locations to something useful by manually creating overrides for them, the rationale behind this patch is not to let these countries appear on productive systems in the first place, as we know they _cannot_ be true.

Therefore, this patch skips any network object that has either AQ or BV country code set.

See also: https://lists.ipfire.org/pipermail/location/2020-October/000199.html

Signed-off-by: Peter Müller peter.mueller@ipfire.org

src/python/location-importer.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/python/location-importer.in b/src/python/location-importer.in index 1e08458..ac678dc 100644 --- a/src/python/location-importer.in +++ b/src/python/location-importer.in @@ -624,7 +624,7 @@ class CLI(object): return

# Skip objects with unknown country codes

• if validcountries and inetnum.get("country") not in validcountries:
  

• if validcountries and (inetnum.get("country") not in validcountries or inetnum.get("country") in ["AQ", "BV"]):
  log.warning("Skipping network with bogus country '%s': %s" % \
  	(inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum")))
  return
  

-- 2.26.2

On 14 May 2021, at 17:15, Peter Müller peter.mueller@ipfire.org wrote:

Hello Michael,

thanks for your reply.

...

Hi,

...

On 12 Apr 2021, at 18:26, Peter Müller peter.mueller@ipfire.org wrote:

Hello Michael,

thanks for your reply.

I believe you are mixing up two distinct scenarios here:

I am trying not to. I just added the name because I struggle to remember some country codes.

...

(a) Historic ISO-3166-X abbreviations for countries no longer existing anymore, such as YU.

They technically do.

They are still reserved in the ISO 3166 database and they show up (in this case) in the RIPE database. So we will have to handle them no matter what.

YU does not have a succeeding national body (like Eastern Germany would have the Federal Republic of Germany) so we cannot automatically rewrite them.

This is what I meant by "does no longer exist anymore".

RIPE seems to think that YU exists :)

As much as I would like it, a standard does not always represent a real-life situation.

...

...

(b) Country codes for existing nations, where we believe with a high level of confidence no IP networks are located in any way, either physically or legally. AQ and BV are the most notable examples known to me at the time of writing.

Since it does not make sense to add historic country codes, I would suggest to leave networks assigned to (a) countries as they are, manually creating overrides for them, as I did before.

For (b), simply scrubbing out the country or region they belong to seems to be too harsh to me, as we use this information for several things, and should not abuse it for our own purposes.

(I assume) we still have AS information about those networks and we should keep at least that.

ACK.

...

...

Therefore, I rather suggest adding an additional column to countries.txt for indicating whether this country code is acceptable to us or not.

Okay. What would you suggest?

Well, our "countries.txt" file's syntax currently looks like (delimited by tabulators):

[ISO-3166-1-alpha2 country code] [regional code defined by us to know which Whois server should be queried] [human-readable name of the country]

I would like to add two new columns to this file, one for declaring a country invalid in terms of libloc, the other one to declare it as suspicious so we can handle those cases better in a later step.

Just for me to check if I understood this correctly:

You would like to add a country like “YU” to the list and if we hit it during the import, we will perform a certain action. What would that action be?

And what is a “suspicious country”? I do not think it would go down well to mark an entire country as such.

Personally, I wish those two columns to be placed before the country name for readability reasons.

It can only be before the country name because country names might have more than one “word” and we cannot split the list correctly any more.

However, this would be a breaking change to existing libloc instances using the "countries.txt" file of our data repository.

Yes. Can we not add markers or something else?

Maybe throw away the CSV format and use something else that can be extended next time?

-Michael

Thanks, and best regards, Peter Müller

...

-Michael

...

Thanks, and best regards, Peter Müller

...

Hello,

How about we add the country to the list and mark the continent with a “-“:

AF AS Afghanistan YU - Yugoslavia

-Michael

...

On 10 Apr 2021, at 13:32, Peter Müller peter.mueller@ipfire.org wrote:

Hello Michael,

thanks for your reply.

Yes, having this configurable in countries.txt would be nice indeed. Do you propose a certain syntax for this?

And yes, a tuple is a better idea here. I will wait for your reply and submit a second version of this patch then.

Thanks, and best regards, Peter Müller

...

Hello,

• would we not want this to be configurable in countries.txt?

• The list should probably be a tuple.

-Michael

> On 1 Apr 2021, at 20:57, Peter Müller peter.mueller@ipfire.org wrote: > > Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the > time of writing. Network owners/operators putting these countries into > their RIR data objects are either completely braindead or doing so for > hostile reasons. > > While we might correct these locations to something useful by manually > creating overrides for them, the rationale behind this patch is not to > let these countries appear on productive systems in the first place, as > we know they _cannot_ be true. > > Therefore, this patch skips any network object that has either AQ or BV > country code set. > > See also: https://lists.ipfire.org/pipermail/location/2020-October/000199.html > > Signed-off-by: Peter Müller peter.mueller@ipfire.org > --- > src/python/location-importer.in | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/src/python/location-importer.in b/src/python/location-importer.in > index 1e08458..ac678dc 100644 > --- a/src/python/location-importer.in > +++ b/src/python/location-importer.in > @@ -624,7 +624,7 @@ class CLI(object): > return > > # Skip objects with unknown country codes > - if validcountries and inetnum.get("country") not in validcountries: > + if validcountries and (inetnum.get("country") not in validcountries or inetnum.get("country") in ["AQ", "BV"]): > log.warning("Skipping network with bogus country '%s': %s" % \ > (inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum"))) > return > -- > 2.26.2