public inbox for location@lists.ipfire.org
 help / color / mirror / Atom feed
* [PATCH] importer: Filter non-global geofeed and RIR umbrella networks
@ 2026-10-18  3:17 Spencer Norton
  2026-10-20 13:18 ` Michael Tremer
  0 siblings, 1 reply; 3+ messages in thread
From: Spencer Norton @ 2026-10-18  3:17 UTC (permalink / raw)
  To: location; +Cc: Michael Tremer, Accounts

Documentation and special-purpose ranges from RFC 5737 (192.0.2.0/24,
198.51.100.0/24, 203.0.113.0/24), RFC 3849 (2001:db8::/32), RFC 5180
(2001:2::/48), RFC 4843 (2001:10::/28), and RFC 3056 (2002::/16) are
currently exported with country: US, and RFC 4380 Teredo (2001::/32) is
exported with aut-num: 6939:

1. _fetch_geofeed() and handle_update_overrides() did not pass parsed
   prefixes through _check_parsed_network(), allowing non-global rows in
   unrestricted geofeeds (such as https://geofeed.constant.com) into
   geofeed_networks. Also reject multicast and reserved ranges in
   _check_parsed_network(), as Python's ipaddress.is_global returns True
   for 224.0.0.0/4 and ff00::/8.
2. handle_write() selected all rows from geofeed_networks into the
   candidate network table n without joining network_geofeeds, emitting
   258 out-of-bailiwick prefixes with no attributes into the database.
3. APNIC publishes top-level IANA pool placeholders (netname: APNIC-AP
   and APNIC-AP-ERX) with country: AU (for APNIC's Brisbane office) across
   its /8 and /12-/23 blocks, which otherwise cover 203.0.113.0/24
   (via 202.0.0.0/7) and 2001:db8::/32 (via 2001:c00::/22) as AU once the
   geofeed entries are removed. Skip APNIC-AP and APNIC-AP-ERX in
   _parse_inetnum_block() alongside the existing RIR umbrella netnames.
4. Purge IPv6 special-purpose ranges (2001::/23, 2001:db8::/32, 3fff::/20)
   from BGP announcements in handle_update_announcements() alongside the
   existing IPv4 test/documentation rules.

Fixes: #13934 - ipfire location db erroneously marks documentation ranges within rfc3849 and rfc5737 belonging to US
Signed-off-by: Spencer Norton <spencer.norton@norvi.tech>
---
 src/scripts/location-importer.in | 30 ++++++++++++++++++++++++++++--
 1 file changed, 28 insertions(+), 2 deletions(-)

diff --git a/src/scripts/location-importer.in b/src/scripts/location-importer.in
index 0298f4f..1a02996 100644
--- a/src/scripts/location-importer.in
+++ b/src/scripts/location-importer.in
@@ -426,9 +426,19 @@ class CLI(object):
 			UNION
 
 			SELECT
-				network
+				geofeed_networks.network
 			FROM
+				geofeeds
+			JOIN
+				network_geofeeds networks
+			ON
+				geofeeds.url = networks.url
+			JOIN
 				geofeed_networks
+			ON
+				geofeeds.id = geofeed_networks.geofeed_id
+			AND
+				networks.network >>= geofeed_networks.network
 		""")
 
 		# Create an index to search through networks faster
@@ -1195,6 +1205,11 @@ class CLI(object):
 			log.debug("Skipping unspecified network: %s" % network)
 			return False
 
+		# Ignore multicast and reserved networks
+		elif network.is_multicast or network.is_reserved:
+			log.debug("Skipping multicast or reserved network: %s" % network)
+			return False
+
 		# IPv6
 		if network.version == 6:
 			if network.prefixlen < 10:
@@ -1319,7 +1334,7 @@ class CLI(object):
 			# Filter any inetnum records which are only referring to IP space
 			# not managed by that specific RIR...
 			if key == "netname":
-				if re.match(r"^(ERX-NETBLOCK|(AFRINIC|ARIN|LACNIC|RIPE)-CIDR-BLOCK|IANA-NETBLOCK-\d{1,3}|NON-RIPE-NCC-MANAGED-ADDRESS-BLOCK|STUB-[\d-]{3,}SLASH\d{1,2})", val.strip()):
+				if re.match(r"^(APNIC-AP(-ERX)?$|ERX-NETBLOCK|(AFRINIC|ARIN|LACNIC|RIPE)-CIDR-BLOCK|IANA-NETBLOCK-\d{1,3}|NON-RIPE-NCC-MANAGED-ADDRESS-BLOCK|STUB-[\d-]{3,}SLASH\d{1,2})", val.strip()):
 					log.debug("Skipping record indicating historic/orphaned data: %s" % val.strip())
 					return
 
@@ -1617,6 +1632,9 @@ class CLI(object):
 				DELETE FROM announcements WHERE family(network) = 4 AND network <<= '198.18.0.0/15';
 				DELETE FROM announcements WHERE family(network) = 4 AND network <<= '198.51.100.0/24';
 				DELETE FROM announcements WHERE family(network) = 4 AND network <<= '203.0.113.0/24';
+				DELETE FROM announcements WHERE family(network) = 6 AND network <<= '2001::/23';
+				DELETE FROM announcements WHERE family(network) = 6 AND network <<= '2001:db8::/32';
+				DELETE FROM announcements WHERE family(network) = 6 AND network <<= '3fff::/20';
 
 				-- DELETE CGNAT address space (RFC 6598)
 				DELETE FROM announcements WHERE family(network) = 4 AND network <<= '100.64.0.0/10';
@@ -1944,6 +1962,10 @@ class CLI(object):
 								log.debug("Could not parse network: %s" % network)
 								continue
 
+							# Sanitize parsed networks...
+							if not self._check_parsed_network(network):
+								continue
+
 							# Strip any excess whitespace from country codes
 							country = country.strip()
 
@@ -2038,6 +2060,10 @@ class CLI(object):
 								log.warning("Skipping %s: You cannot overwrite default" % network)
 								continue
 
+							# Sanitize parsed networks...
+							if not self._check_parsed_network(network):
+								continue
+
 							self.db.execute("""
 								INSERT INTO
 									network_overrides
-- 
2.50.1



^ permalink raw reply	[flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-10-20 15:47 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-10-18  3:17 [PATCH] importer: Filter non-global geofeed and RIR umbrella networks Spencer Norton
2026-10-20 13:18 ` Michael Tremer
2026-10-20 15:45   ` Spencer Norton

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox