From mboxrd@z Thu Jan  1 00:00:00 1970
From: Peter =?utf-8?q?M=C3=BCller?= <peter.mueller@ipfire.org>
To: location@lists.ipfire.org
Subject: Re: [PATCH] location-importer.in: Attempt to provide meaningful AS
 names if organisation handles are missing
Date: Wed, 21 Jul 2021 23:23:25 +0200
Message-ID: <257789bc-404e-dc5c-7e6f-41a1479d6ad3@ipfire.org>
In-Reply-To: <7DFE76D5-572D-4D40-9AA3-CFF79D28AD6E@ipfire.org>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============1478208105757323727=="
List-Id: <location.lists.ipfire.org>

--===============1478208105757323727==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Hello Michael,

thanks for your reply.

I would leave this up to you - some data-related issues are still on my list,=
 but I
am unfortunately really short on spare time at the moment. So I guess we shou=
ld get
in production what we have at hand... :-)

By the way: Thank you for fixing location01, SSH login works fine for me agai=
n. :-)

Thanks, and best regards,
Peter M=C3=BCller


> Thank you. Merged.
>=20
> Do we plan a release because of this soon?
>=20
>> On 19 Jul 2021, at 22:34, Peter M=C3=BCller <peter.mueller(a)ipfire.org> w=
rote:
>>
>> A decent amount of autnum objects - especially, but not exclusively in
>> the APNIC sector - does not contain a link to an organisation handle.
>>
>> In such cases, this patch is going to use the first description line of
>> the atunum object in question (if available) as a string for its name.
>> The overwhelming majority of affected ASNs contains a valuable
>> information there, so this is almost as good as having an organisation
>> handle linked to it.
>>
>> Fixes: #12660
>>
>> Signed-off-by: Peter M=C3=BCller <peter.mueller(a)ipfire.org>
>> ---
>> src/python/location-importer.in | 28 +++++++++++++++++++++++++---
>> 1 file changed, 25 insertions(+), 3 deletions(-)
>>
>> diff --git a/src/python/location-importer.in b/src/python/location-importe=
r.in
>> index d4b0ca2..bec0b62 100644
>> --- a/src/python/location-importer.in
>> +++ b/src/python/location-importer.in
>> @@ -393,11 +393,11 @@ class CLI(object):
>> 		with self.db.transaction():
>> 			# Create some temporary tables to store parsed data
>> 			self.db.execute("""
>> -				CREATE TEMPORARY TABLE _autnums(number integer, organization text, so=
urce text NOT NULL)
>> +				CREATE TEMPORARY TABLE _autnums(number integer NOT NULL, organization=
 text NOT NULL, source text NOT NULL)
>> 					ON COMMIT DROP;
>> 				CREATE UNIQUE INDEX _autnums_number ON _autnums(number);
>>
>> -				CREATE TEMPORARY TABLE _organizations(handle text, name text NOT NULL=
, source text NOT NULL)
>> +				CREATE TEMPORARY TABLE _organizations(handle text NOT NULL, name text=
 NOT NULL, source text NOT NULL)
>> 					ON COMMIT DROP;
>> 				CREATE UNIQUE INDEX _organizations_handle ON _organizations(handle);
>>
>> @@ -604,10 +604,32 @@ class CLI(object):
>> 			elif key =3D=3D "org":
>> 				autnum[key] =3D val.upper()
>>
>> +			elif key =3D=3D "descr":
>> +				# Save the first description line as well...
>> +				if not key in autnum:
>> +					autnum[key] =3D val
>> +
>> 		# Skip empty objects
>> -		if not autnum:
>> +		if not autnum or not "asn" in autnum:
>> 			return
>>
>> +		# Insert a dummy organisation handle into our temporary organisations
>> +		# table in case the AS does not have an organisation handle set, but
>> +		# has a description (a quirk often observed in APNIC area), so we can
>> +		# later display at least some string for this AS.
>> +		if not "org" in autnum:
>> +			if "descr" in autnum:
>> +				autnum["org"] =3D "LIBLOC-%s-ORGHANDLE" % autnum.get("asn")
>> +
>> +				self.db.execute("INSERT INTO _organizations(handle, name, source) \
>> +					VALUES(%s, %s, %s) ON CONFLICT (handle) DO NOTHING",
>> +					autnum.get("org"), autnum.get("descr"), source_key,
>> +				)
>> +			else:
>> +				log.warning("ASN %s neither has an organisation handle nor a descript=
ion line set, omitting" % \
>> +						autnum.get("asn"))
>> +				return
>> +
>> 		# Insert into database
>> 		self.db.execute("INSERT INTO _autnums(number, organization, source) \
>> 			VALUES(%s, %s, %s) ON CONFLICT (number) DO UPDATE SET \
>> --=20
>> 2.20.1
>>
>=20

--===============1478208105757323727==--