From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter =?utf-8?q?M=C3=BCller?= To: location@lists.ipfire.org Subject: Re: [PATCH] location-importer.in: treat AQ and BV as invalid countries Date: Fri, 14 May 2021 18:15:24 +0200 Message-ID: <1c89134e-4f5b-4654-3443-ea19fc225e8e@ipfire.org> In-Reply-To: <32E9519B-895B-4514-A14B-5A9307A5907D@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4412754138622658088==" List-Id: --===============4412754138622658088== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello Michael, thanks for your reply. > Hi, >=20 >> On 12 Apr 2021, at 18:26, Peter M=C3=BCller w= rote: >> >> Hello Michael, >> >> thanks for your reply. >> >> I believe you are mixing up two distinct scenarios here: >=20 > I am trying not to. I just added the name because I struggle to remember so= me country codes. >=20 >> (a) Historic ISO-3166-X abbreviations for countries no longer existing any= more, such as YU. >=20 > They technically do. >=20 > They are still reserved in the ISO 3166 database and they show up (in this = case) in the RIPE database. So we will have to handle them no matter what. >=20 > YU does not have a succeeding national body (like Eastern Germany would hav= e the Federal Republic of Germany) so we cannot automatically rewrite them. This is what I meant by "does no longer exist anymore". >=20 >> (b) Country codes for existing nations, where we believe with a high level= of confidence no >> IP networks are located in any way, either physically or legally. AQ an= d BV are the most >> notable examples known to me at the time of writing. >> >> Since it does not make sense to add historic country codes, I would sugges= t to leave networks >> assigned to (a) countries as they are, manually creating overrides for the= m, as I did before. >> >> For (b), simply scrubbing out the country or region they belong to seems t= o be too harsh to >> me, as we use this information for several things, and should not abuse it= for our own purposes. >=20 > (I assume) we still have AS information about those networks and we should = keep at least that. ACK. >=20 >> Therefore, I rather suggest adding an additional column to countries.txt f= or indicating whether >> this country code is acceptable to us or not. >=20 > Okay. What would you suggest? Well, our "countries.txt" file's syntax currently looks like (delimited by ta= bulators): [ISO-3166-1-alpha2 country code] [regional code defined by us to know which W= hois server should be queried] [human-readable name of the country] I would like to add two new columns to this file, one for declaring a country= invalid in terms of libloc, the other one to declare it as suspicious so we can handle those case= s better in a later step. Personally, I wish those two columns to be placed before the country name for= readability reasons. However, this would be a breaking change to existing libloc instances using t= he "countries.txt" file of our data repository. Thanks, and best regards, Peter M=C3=BCller >=20 > -Michael >=20 >> >> Thanks, and best regards, >> Peter M=C3=BCller >> >>> Hello, >>> >>> How about we add the country to the list and mark the continent with a = =E2=80=9C-=E2=80=9C: >>> >>> AF AS Afghanistan >>> YU - Yugoslavia >>> >>> -Michael >>> >>>> On 10 Apr 2021, at 13:32, Peter M=C3=BCller = wrote: >>>> >>>> Hello Michael, >>>> >>>> thanks for your reply. >>>> >>>> Yes, having this configurable in countries.txt would be nice indeed. Do = you propose a certain syntax for this? >>>> >>>> And yes, a tuple is a better idea here. I will wait for your reply and s= ubmit a second version of this patch then. >>>> >>>> Thanks, and best regards, >>>> Peter M=C3=BCller >>>> >>>> >>>>> Hello, >>>>> >>>>> * would we not want this to be configurable in countries.txt? >>>>> >>>>> * The list should probably be a tuple. >>>>> >>>>> -Michael >>>>> >>>>>> On 1 Apr 2021, at 20:57, Peter M=C3=BCller wrote: >>>>>> >>>>>> Both the Bouvet Island (BV) and Antarctica (AQ) are unpopulated at the >>>>>> time of writing. Network owners/operators putting these countries into >>>>>> their RIR data objects are either completely braindead or doing so for >>>>>> hostile reasons. >>>>>> >>>>>> While we might correct these locations to something useful by manually >>>>>> creating overrides for them, the rationale behind this patch is not to >>>>>> let these countries appear on productive systems in the first place, as >>>>>> we know they _cannot_ be true. >>>>>> >>>>>> Therefore, this patch skips any network object that has either AQ or BV >>>>>> country code set. >>>>>> >>>>>> See also: https://lists.ipfire.org/pipermail/location/2020-October/000= 199.html >>>>>> >>>>>> Signed-off-by: Peter M=C3=BCller >>>>>> --- >>>>>> src/python/location-importer.in | 2 +- >>>>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>>>> >>>>>> diff --git a/src/python/location-importer.in b/src/python/location-imp= orter.in >>>>>> index 1e08458..ac678dc 100644 >>>>>> --- a/src/python/location-importer.in >>>>>> +++ b/src/python/location-importer.in >>>>>> @@ -624,7 +624,7 @@ class CLI(object): >>>>>> return >>>>>> >>>>>> # Skip objects with unknown country codes >>>>>> - if validcountries and inetnum.get("country") not in validcountries: >>>>>> + if validcountries and (inetnum.get("country") not in validcountries= or inetnum.get("country") in ["AQ", "BV"]): >>>>>> log.warning("Skipping network with bogus country '%s': %s" % \ >>>>>> (inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("i= netnum"))) >>>>>> return >>>>>> --=20 >>>>>> 2.26.2 >>>>> >>> >=20 --===============4412754138622658088==--