[PATCH] location-importer.in: treat AQ and BV as invalid countries

[PATCH] location-importer.in: treat AQ and BV as invalid countries

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 1472 bytes --]

Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the
time of writing. Network owners/operators putting these countries into
their RIR data objects are either completely braindead or doing so for
hostile reasons.

While we might correct these locations to something useful by manually
creating overrides for them, the rationale behind this patch is not to
let these countries appear on productive systems in the first place, as
we know they _cannot_ be true.

Therefore, this patch skips any network object that has either AQ or BV
country code set.

See also: https://lists.ipfire.org/pipermail/location/2020-October/000199.html

Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org>
---
 src/python/location-importer.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/python/location-importer.in b/src/python/location-importer.in
index 1e08458..ac678dc 100644
--- a/src/python/location-importer.in
+++ b/src/python/location-importer.in
@@ -624,7 +624,7 @@ class CLI(object):
 			return
 
 		# Skip objects with unknown country codes
-		if validcountries and inetnum.get("country") not in validcountries:
+		if validcountries and (inetnum.get("country") not in validcountries or inetnum.get("country") in ["AQ", "BV"]):
 			log.warning("Skipping network with bogus country '%s': %s" % \
 				(inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum")))
 			return
-- 
2.26.2

Re: [PATCH] location-importer.in: treat AQ and BV as invalid countries

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 1747 bytes --]

Hello,

* would we not want this to be configurable in countries.txt?

* The list should probably be a tuple.

-Michael

> On 1 Apr 2021, at 20:57, Peter Müller <peter.mueller(a)ipfire.org> wrote:
> 
> Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the
> time of writing. Network owners/operators putting these countries into
> their RIR data objects are either completely braindead or doing so for
> hostile reasons.
> 
> While we might correct these locations to something useful by manually
> creating overrides for them, the rationale behind this patch is not to
> let these countries appear on productive systems in the first place, as
> we know they _cannot_ be true.
> 
> Therefore, this patch skips any network object that has either AQ or BV
> country code set.
> 
> See also: https://lists.ipfire.org/pipermail/location/2020-October/000199.html
> 
> Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org>
> ---
> src/python/location-importer.in | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/src/python/location-importer.in b/src/python/location-importer.in
> index 1e08458..ac678dc 100644
> --- a/src/python/location-importer.in
> +++ b/src/python/location-importer.in
> @@ -624,7 +624,7 @@ class CLI(object):
> 			return
> 
> 		# Skip objects with unknown country codes
> -		if validcountries and inetnum.get("country") not in validcountries:
> +		if validcountries and (inetnum.get("country") not in validcountries or inetnum.get("country") in ["AQ", "BV"]):
> 			log.warning("Skipping network with bogus country '%s': %s" % \
> 				(inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum")))
> 			return
> -- 
> 2.26.2

Re: [PATCH] location-importer.in: treat AQ and BV as invalid countries

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 2117 bytes --]

Hello Michael,

thanks for your reply.

Yes, having this configurable in countries.txt would be nice indeed. Do you propose a certain syntax for this?

And yes, a tuple is a better idea here. I will wait for your reply and submit a second version of this patch then.

Thanks, and best regards,
Peter Müller


> Hello,
> 
> * would we not want this to be configurable in countries.txt?
> 
> * The list should probably be a tuple.
> 
> -Michael
> 
>> On 1 Apr 2021, at 20:57, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>
>> Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the
>> time of writing. Network owners/operators putting these countries into
>> their RIR data objects are either completely braindead or doing so for
>> hostile reasons.
>>
>> While we might correct these locations to something useful by manually
>> creating overrides for them, the rationale behind this patch is not to
>> let these countries appear on productive systems in the first place, as
>> we know they _cannot_ be true.
>>
>> Therefore, this patch skips any network object that has either AQ or BV
>> country code set.
>>
>> See also: https://lists.ipfire.org/pipermail/location/2020-October/000199.html
>>
>> Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org>
>> ---
>> src/python/location-importer.in | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/src/python/location-importer.in b/src/python/location-importer.in
>> index 1e08458..ac678dc 100644
>> --- a/src/python/location-importer.in
>> +++ b/src/python/location-importer.in
>> @@ -624,7 +624,7 @@ class CLI(object):
>> 			return
>>
>> 		# Skip objects with unknown country codes
>> -		if validcountries and inetnum.get("country") not in validcountries:
>> +		if validcountries and (inetnum.get("country") not in validcountries or inetnum.get("country") in ["AQ", "BV"]):
>> 			log.warning("Skipping network with bogus country '%s': %s" % \
>> 				(inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum")))
>> 			return
>> -- 
>> 2.26.2
> 

Re: [PATCH] location-importer.in: treat AQ and BV as invalid countries

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 2447 bytes --]

Hello,

How about we add the country to the list and mark the continent with a “-“:

  AF      AS      Afghanistan
  YU      -       Yugoslavia

-Michael

> On 10 Apr 2021, at 13:32, Peter Müller <peter.mueller(a)ipfire.org> wrote:
> 
> Hello Michael,
> 
> thanks for your reply.
> 
> Yes, having this configurable in countries.txt would be nice indeed. Do you propose a certain syntax for this?
> 
> And yes, a tuple is a better idea here. I will wait for your reply and submit a second version of this patch then.
> 
> Thanks, and best regards,
> Peter Müller
> 
> 
>> Hello,
>> 
>> * would we not want this to be configurable in countries.txt?
>> 
>> * The list should probably be a tuple.
>> 
>> -Michael
>> 
>>> On 1 Apr 2021, at 20:57, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>> 
>>> Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the
>>> time of writing. Network owners/operators putting these countries into
>>> their RIR data objects are either completely braindead or doing so for
>>> hostile reasons.
>>> 
>>> While we might correct these locations to something useful by manually
>>> creating overrides for them, the rationale behind this patch is not to
>>> let these countries appear on productive systems in the first place, as
>>> we know they _cannot_ be true.
>>> 
>>> Therefore, this patch skips any network object that has either AQ or BV
>>> country code set.
>>> 
>>> See also: https://lists.ipfire.org/pipermail/location/2020-October/000199.html
>>> 
>>> Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org>
>>> ---
>>> src/python/location-importer.in | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>> 
>>> diff --git a/src/python/location-importer.in b/src/python/location-importer.in
>>> index 1e08458..ac678dc 100644
>>> --- a/src/python/location-importer.in
>>> +++ b/src/python/location-importer.in
>>> @@ -624,7 +624,7 @@ class CLI(object):
>>> 			return
>>> 
>>> 		# Skip objects with unknown country codes
>>> -		if validcountries and inetnum.get("country") not in validcountries:
>>> +		if validcountries and (inetnum.get("country") not in validcountries or inetnum.get("country") in ["AQ", "BV"]):
>>> 			log.warning("Skipping network with bogus country '%s': %s" % \
>>> 				(inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum")))
>>> 			return
>>> -- 
>>> 2.26.2
>> 

Re: [PATCH] location-importer.in: treat AQ and BV as invalid countries

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 3527 bytes --]

Hello Michael,

thanks for your reply.

I believe you are mixing up two distinct scenarios here:

(a) Historic ISO-3166-X abbreviations for countries no longer existing anymore, such as YU.

(b) Country codes for existing nations, where we believe with a high level of confidence no
    IP networks are located in any way, either physically or legally. AQ and BV are the most
    notable examples known to me at the time of writing.

Since it does not make sense to add historic country codes, I would suggest to leave networks
assigned to (a) countries as they are, manually creating overrides for them, as I did before.

For (b), simply scrubbing out the country or region they belong to seems to be too harsh to
me, as we use this information for several things, and should not abuse it for our own purposes.
Therefore, I rather suggest adding an additional column to countries.txt for indicating whether
this country code is acceptable to us or not.

Thanks, and best regards,
Peter Müller

> Hello,
> 
> How about we add the country to the list and mark the continent with a “-“:
> 
>   AF      AS      Afghanistan
>   YU      -       Yugoslavia
> 
> -Michael
> 
>> On 10 Apr 2021, at 13:32, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>
>> Hello Michael,
>>
>> thanks for your reply.
>>
>> Yes, having this configurable in countries.txt would be nice indeed. Do you propose a certain syntax for this?
>>
>> And yes, a tuple is a better idea here. I will wait for your reply and submit a second version of this patch then.
>>
>> Thanks, and best regards,
>> Peter Müller
>>
>>
>>> Hello,
>>>
>>> * would we not want this to be configurable in countries.txt?
>>>
>>> * The list should probably be a tuple.
>>>
>>> -Michael
>>>
>>>> On 1 Apr 2021, at 20:57, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>>>
>>>> Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the
>>>> time of writing. Network owners/operators putting these countries into
>>>> their RIR data objects are either completely braindead or doing so for
>>>> hostile reasons.
>>>>
>>>> While we might correct these locations to something useful by manually
>>>> creating overrides for them, the rationale behind this patch is not to
>>>> let these countries appear on productive systems in the first place, as
>>>> we know they _cannot_ be true.
>>>>
>>>> Therefore, this patch skips any network object that has either AQ or BV
>>>> country code set.
>>>>
>>>> See also: https://lists.ipfire.org/pipermail/location/2020-October/000199.html
>>>>
>>>> Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org>
>>>> ---
>>>> src/python/location-importer.in | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/src/python/location-importer.in b/src/python/location-importer.in
>>>> index 1e08458..ac678dc 100644
>>>> --- a/src/python/location-importer.in
>>>> +++ b/src/python/location-importer.in
>>>> @@ -624,7 +624,7 @@ class CLI(object):
>>>> 			return
>>>>
>>>> 		# Skip objects with unknown country codes
>>>> -		if validcountries and inetnum.get("country") not in validcountries:
>>>> +		if validcountries and (inetnum.get("country") not in validcountries or inetnum.get("country") in ["AQ", "BV"]):
>>>> 			log.warning("Skipping network with bogus country '%s': %s" % \
>>>> 				(inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum")))
>>>> 			return
>>>> -- 
>>>> 2.26.2
>>>
> 

Re: [PATCH] location-importer.in: treat AQ and BV as invalid countries

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 4320 bytes --]

Hi,

> On 12 Apr 2021, at 18:26, Peter Müller <peter.mueller(a)ipfire.org> wrote:
> 
> Hello Michael,
> 
> thanks for your reply.
> 
> I believe you are mixing up two distinct scenarios here:

I am trying not to. I just added the name because I struggle to remember some country codes.

> (a) Historic ISO-3166-X abbreviations for countries no longer existing anymore, such as YU.

They technically do.

They are still reserved in the ISO 3166 database and they show up (in this case) in the RIPE database. So we will have to handle them no matter what.

YU does not have a succeeding national body (like Eastern Germany would have the Federal Republic of Germany) so we cannot automatically rewrite them.

> (b) Country codes for existing nations, where we believe with a high level of confidence no
>    IP networks are located in any way, either physically or legally. AQ and BV are the most
>    notable examples known to me at the time of writing.
> 
> Since it does not make sense to add historic country codes, I would suggest to leave networks
> assigned to (a) countries as they are, manually creating overrides for them, as I did before.
> 
> For (b), simply scrubbing out the country or region they belong to seems to be too harsh to
> me, as we use this information for several things, and should not abuse it for our own purposes.

(I assume) we still have AS information about those networks and we should keep at least that.

> Therefore, I rather suggest adding an additional column to countries.txt for indicating whether
> this country code is acceptable to us or not.

Okay. What would you suggest?

-Michael

> 
> Thanks, and best regards,
> Peter Müller
> 
>> Hello,
>> 
>> How about we add the country to the list and mark the continent with a “-“:
>> 
>>  AF      AS      Afghanistan
>>  YU      -       Yugoslavia
>> 
>> -Michael
>> 
>>> On 10 Apr 2021, at 13:32, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>> 
>>> Hello Michael,
>>> 
>>> thanks for your reply.
>>> 
>>> Yes, having this configurable in countries.txt would be nice indeed. Do you propose a certain syntax for this?
>>> 
>>> And yes, a tuple is a better idea here. I will wait for your reply and submit a second version of this patch then.
>>> 
>>> Thanks, and best regards,
>>> Peter Müller
>>> 
>>> 
>>>> Hello,
>>>> 
>>>> * would we not want this to be configurable in countries.txt?
>>>> 
>>>> * The list should probably be a tuple.
>>>> 
>>>> -Michael
>>>> 
>>>>> On 1 Apr 2021, at 20:57, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>>>> 
>>>>> Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the
>>>>> time of writing. Network owners/operators putting these countries into
>>>>> their RIR data objects are either completely braindead or doing so for
>>>>> hostile reasons.
>>>>> 
>>>>> While we might correct these locations to something useful by manually
>>>>> creating overrides for them, the rationale behind this patch is not to
>>>>> let these countries appear on productive systems in the first place, as
>>>>> we know they _cannot_ be true.
>>>>> 
>>>>> Therefore, this patch skips any network object that has either AQ or BV
>>>>> country code set.
>>>>> 
>>>>> See also: https://lists.ipfire.org/pipermail/location/2020-October/000199.html
>>>>> 
>>>>> Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org>
>>>>> ---
>>>>> src/python/location-importer.in | 2 +-
>>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>> 
>>>>> diff --git a/src/python/location-importer.in b/src/python/location-importer.in
>>>>> index 1e08458..ac678dc 100644
>>>>> --- a/src/python/location-importer.in
>>>>> +++ b/src/python/location-importer.in
>>>>> @@ -624,7 +624,7 @@ class CLI(object):
>>>>> 			return
>>>>> 
>>>>> 		# Skip objects with unknown country codes
>>>>> -		if validcountries and inetnum.get("country") not in validcountries:
>>>>> +		if validcountries and (inetnum.get("country") not in validcountries or inetnum.get("country") in ["AQ", "BV"]):
>>>>> 			log.warning("Skipping network with bogus country '%s': %s" % \
>>>>> 				(inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum")))
>>>>> 			return
>>>>> -- 
>>>>> 2.26.2
>>>> 
>> 

Re: [PATCH] location-importer.in: treat AQ and BV as invalid countries

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 5259 bytes --]

Hello Michael,

thanks for your reply.

> Hi,
> 
>> On 12 Apr 2021, at 18:26, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>
>> Hello Michael,
>>
>> thanks for your reply.
>>
>> I believe you are mixing up two distinct scenarios here:
> 
> I am trying not to. I just added the name because I struggle to remember some country codes.
> 
>> (a) Historic ISO-3166-X abbreviations for countries no longer existing anymore, such as YU.
> 
> They technically do.
> 
> They are still reserved in the ISO 3166 database and they show up (in this case) in the RIPE database. So we will have to handle them no matter what.
> 
> YU does not have a succeeding national body (like Eastern Germany would have the Federal Republic of Germany) so we cannot automatically rewrite them.

This is what I meant by "does no longer exist anymore".

> 
>> (b) Country codes for existing nations, where we believe with a high level of confidence no
>>    IP networks are located in any way, either physically or legally. AQ and BV are the most
>>    notable examples known to me at the time of writing.
>>
>> Since it does not make sense to add historic country codes, I would suggest to leave networks
>> assigned to (a) countries as they are, manually creating overrides for them, as I did before.
>>
>> For (b), simply scrubbing out the country or region they belong to seems to be too harsh to
>> me, as we use this information for several things, and should not abuse it for our own purposes.
> 
> (I assume) we still have AS information about those networks and we should keep at least that.

ACK.

> 
>> Therefore, I rather suggest adding an additional column to countries.txt for indicating whether
>> this country code is acceptable to us or not.
> 
> Okay. What would you suggest?

Well, our "countries.txt" file's syntax currently looks like (delimited by tabulators):

[ISO-3166-1-alpha2 country code]	[regional code defined by us to know which Whois server should be queried]	[human-readable name of the country]

I would like to add two new columns to this file, one for declaring a country invalid in terms of
libloc, the other one to declare it as suspicious so we can handle those cases better in a later
step.

Personally, I wish those two columns to be placed before the country name for readability reasons.
However, this would be a breaking change to existing libloc instances using the "countries.txt" file
of our data repository.

Thanks, and best regards,
Peter Müller

> 
> -Michael
> 
>>
>> Thanks, and best regards,
>> Peter Müller
>>
>>> Hello,
>>>
>>> How about we add the country to the list and mark the continent with a “-“:
>>>
>>>  AF      AS      Afghanistan
>>>  YU      -       Yugoslavia
>>>
>>> -Michael
>>>
>>>> On 10 Apr 2021, at 13:32, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>>>
>>>> Hello Michael,
>>>>
>>>> thanks for your reply.
>>>>
>>>> Yes, having this configurable in countries.txt would be nice indeed. Do you propose a certain syntax for this?
>>>>
>>>> And yes, a tuple is a better idea here. I will wait for your reply and submit a second version of this patch then.
>>>>
>>>> Thanks, and best regards,
>>>> Peter Müller
>>>>
>>>>
>>>>> Hello,
>>>>>
>>>>> * would we not want this to be configurable in countries.txt?
>>>>>
>>>>> * The list should probably be a tuple.
>>>>>
>>>>> -Michael
>>>>>
>>>>>> On 1 Apr 2021, at 20:57, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>>>>>
>>>>>> Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the
>>>>>> time of writing. Network owners/operators putting these countries into
>>>>>> their RIR data objects are either completely braindead or doing so for
>>>>>> hostile reasons.
>>>>>>
>>>>>> While we might correct these locations to something useful by manually
>>>>>> creating overrides for them, the rationale behind this patch is not to
>>>>>> let these countries appear on productive systems in the first place, as
>>>>>> we know they _cannot_ be true.
>>>>>>
>>>>>> Therefore, this patch skips any network object that has either AQ or BV
>>>>>> country code set.
>>>>>>
>>>>>> See also: https://lists.ipfire.org/pipermail/location/2020-October/000199.html
>>>>>>
>>>>>> Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org>
>>>>>> ---
>>>>>> src/python/location-importer.in | 2 +-
>>>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/src/python/location-importer.in b/src/python/location-importer.in
>>>>>> index 1e08458..ac678dc 100644
>>>>>> --- a/src/python/location-importer.in
>>>>>> +++ b/src/python/location-importer.in
>>>>>> @@ -624,7 +624,7 @@ class CLI(object):
>>>>>> 			return
>>>>>>
>>>>>> 		# Skip objects with unknown country codes
>>>>>> -		if validcountries and inetnum.get("country") not in validcountries:
>>>>>> +		if validcountries and (inetnum.get("country") not in validcountries or inetnum.get("country") in ["AQ", "BV"]):
>>>>>> 			log.warning("Skipping network with bogus country '%s': %s" % \
>>>>>> 				(inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum")))
>>>>>> 			return
>>>>>> -- 
>>>>>> 2.26.2
>>>>>
>>>
> 

Re: [PATCH] location-importer.in: treat AQ and BV as invalid countries

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 6278 bytes --]



> On 14 May 2021, at 17:15, Peter Müller <peter.mueller(a)ipfire.org> wrote:
> 
> Hello Michael,
> 
> thanks for your reply.
> 
>> Hi,
>> 
>>> On 12 Apr 2021, at 18:26, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>> 
>>> Hello Michael,
>>> 
>>> thanks for your reply.
>>> 
>>> I believe you are mixing up two distinct scenarios here:
>> 
>> I am trying not to. I just added the name because I struggle to remember some country codes.
>> 
>>> (a) Historic ISO-3166-X abbreviations for countries no longer existing anymore, such as YU.
>> 
>> They technically do.
>> 
>> They are still reserved in the ISO 3166 database and they show up (in this case) in the RIPE database. So we will have to handle them no matter what.
>> 
>> YU does not have a succeeding national body (like Eastern Germany would have the Federal Republic of Germany) so we cannot automatically rewrite them.
> 
> This is what I meant by "does no longer exist anymore".

RIPE seems to think that YU exists :)

As much as I would like it, a standard does not always represent a real-life situation.

>> 
>>> (b) Country codes for existing nations, where we believe with a high level of confidence no
>>>   IP networks are located in any way, either physically or legally. AQ and BV are the most
>>>   notable examples known to me at the time of writing.
>>> 
>>> Since it does not make sense to add historic country codes, I would suggest to leave networks
>>> assigned to (a) countries as they are, manually creating overrides for them, as I did before.
>>> 
>>> For (b), simply scrubbing out the country or region they belong to seems to be too harsh to
>>> me, as we use this information for several things, and should not abuse it for our own purposes.
>> 
>> (I assume) we still have AS information about those networks and we should keep at least that.
> 
> ACK.
> 
>> 
>>> Therefore, I rather suggest adding an additional column to countries.txt for indicating whether
>>> this country code is acceptable to us or not.
>> 
>> Okay. What would you suggest?
> 
> Well, our "countries.txt" file's syntax currently looks like (delimited by tabulators):
> 
> [ISO-3166-1-alpha2 country code]	[regional code defined by us to know which Whois server should be queried]	[human-readable name of the country]
> 
> I would like to add two new columns to this file, one for declaring a country invalid in terms of
> libloc, the other one to declare it as suspicious so we can handle those cases better in a later
> step.

Just for me to check if I understood this correctly:

You would like to add a country like “YU” to the list and if we hit it during the import, we will perform a certain action. What would that action be?

And what is a “suspicious country”? I do not think it would go down well to mark an entire country as such.

> Personally, I wish those two columns to be placed before the country name for readability reasons.

It can only be before the country name because country names might have more than one “word” and we cannot split the list correctly any more.

> However, this would be a breaking change to existing libloc instances using the "countries.txt" file
> of our data repository.

Yes. Can we not add markers or something else?

Maybe throw away the CSV format and use something else that can be extended next time?

-Michael

> Thanks, and best regards,
> Peter Müller
> 
>> 
>> -Michael
>> 
>>> 
>>> Thanks, and best regards,
>>> Peter Müller
>>> 
>>>> Hello,
>>>> 
>>>> How about we add the country to the list and mark the continent with a “-“:
>>>> 
>>>> AF      AS      Afghanistan
>>>> YU      -       Yugoslavia
>>>> 
>>>> -Michael
>>>> 
>>>>> On 10 Apr 2021, at 13:32, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>>>> 
>>>>> Hello Michael,
>>>>> 
>>>>> thanks for your reply.
>>>>> 
>>>>> Yes, having this configurable in countries.txt would be nice indeed. Do you propose a certain syntax for this?
>>>>> 
>>>>> And yes, a tuple is a better idea here. I will wait for your reply and submit a second version of this patch then.
>>>>> 
>>>>> Thanks, and best regards,
>>>>> Peter Müller
>>>>> 
>>>>> 
>>>>>> Hello,
>>>>>> 
>>>>>> * would we not want this to be configurable in countries.txt?
>>>>>> 
>>>>>> * The list should probably be a tuple.
>>>>>> 
>>>>>> -Michael
>>>>>> 
>>>>>>> On 1 Apr 2021, at 20:57, Peter Müller <peter.mueller(a)ipfire.org> wrote:
>>>>>>> 
>>>>>>> Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the
>>>>>>> time of writing. Network owners/operators putting these countries into
>>>>>>> their RIR data objects are either completely braindead or doing so for
>>>>>>> hostile reasons.
>>>>>>> 
>>>>>>> While we might correct these locations to something useful by manually
>>>>>>> creating overrides for them, the rationale behind this patch is not to
>>>>>>> let these countries appear on productive systems in the first place, as
>>>>>>> we know they _cannot_ be true.
>>>>>>> 
>>>>>>> Therefore, this patch skips any network object that has either AQ or BV
>>>>>>> country code set.
>>>>>>> 
>>>>>>> See also: https://lists.ipfire.org/pipermail/location/2020-October/000199.html
>>>>>>> 
>>>>>>> Signed-off-by: Peter Müller <peter.mueller(a)ipfire.org>
>>>>>>> ---
>>>>>>> src/python/location-importer.in | 2 +-
>>>>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>> 
>>>>>>> diff --git a/src/python/location-importer.in b/src/python/location-importer.in
>>>>>>> index 1e08458..ac678dc 100644
>>>>>>> --- a/src/python/location-importer.in
>>>>>>> +++ b/src/python/location-importer.in
>>>>>>> @@ -624,7 +624,7 @@ class CLI(object):
>>>>>>> 			return
>>>>>>> 
>>>>>>> 		# Skip objects with unknown country codes
>>>>>>> -		if validcountries and inetnum.get("country") not in validcountries:
>>>>>>> +		if validcountries and (inetnum.get("country") not in validcountries or inetnum.get("country") in ["AQ", "BV"]):
>>>>>>> 			log.warning("Skipping network with bogus country '%s': %s" % \
>>>>>>> 				(inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum")))
>>>>>>> 			return
>>>>>>> -- 
>>>>>>> 2.26.2