Re: Reduced AS/route visibility

[Michael Tremer <michael.tremer@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 3495 bytes --]

Hello Jordan,

Thank you very much for your email. This has really helped me.

Recently I have been rolling out a lot of changes that massively improve the database. I have not been 100% confident that all problems have been solved, however I did not have and indication that things went wrong.

One of the changes was a deduplication algorithm that is supposed to remove any subnets from the database we don’t need. That works well.

The second change is to merge neighbouring subnets (e.g. 10.0.0.0/24 and 10.0.1.0/24 could be stored as 10.0.0.0/23). That would save a lot of space, too. That algorithm relied on a function to count the bit length of an IP address (i.e. the minimum number of bits I need to represent a certain IP address). That function was total garbage. No idea how that didn’t cause bigger damage.

The fix is now here:

  https://git.ipfire.org/?p=location/libloc.git;a=commitdiff;h=edbf280e6e043eaf38d4526e421cd68fcf7bc5c0

I have published an updated version of the database with the fix:

  https://git.ipfire.org/?p=location/location-database.git;a=commitdiff;h=16e9065d8017548999e4c4817206214613cd1e02

We now have data for the networks that you have listed (as expected):

root(a)michael:~# location lookup 209.38.204.47 45.148.122.0 94.16.116.0 198.23.234.0 172.105.192.0
209.38.204.47:
  Network                 : 209.38.192.0/18
  Country                 : United States of America
  Autonomous System       : AS14061 - DigitalOcean, LLC
45.148.122.0:
  Network                 : 45.148.122.0/24
  Country                 : Netherlands
  Autonomous System       : AS64425 - SKB Enterprise B.V.
  Hostile Network safe to drop: yes
94.16.116.0:
  Network                 : 94.16.112.0/21
  Country                 : Germany
  Autonomous System       : AS197540 - netcup GmbH
198.23.234.0:
  Network                 : 198.23.224.0/20
  Country                 : Canada
  Autonomous System       : AS36352 - HostPapa
172.105.192.0:
  Network                 : 172.105.0.0/16
  Country                 : United States of America
  Autonomous System       : AS63949 - Akamai Technologies, Inc.

This all matches the table that you have drawn.

So, thank you very much for helping me find this bug. It is solved!

Best,
-Michael

P.S. Out of my own curiosity, may I ask what your application for this database is?

> On 22 Mar 2024, at 04:03, Jordan Savoca <jsavoca(a)posteo.net> wrote:
> 
> It seems there has been a significant reduction in the visibility of
> certain routes in the IPFire dataset recently, and/or spurious routes
> taking place of legitimate announcements.
> 
> For example, 209.38.204.47 is originated by AS14061 and announced in
> 209.38.192.0/19, but IPFire erroneously sees 209.38.0.0/16 and locates
> no corresponding AS.
> 
> Here are a few more prefixes with the same issue:
> 
> +----------+------------------+-----------------+------------+
> |   ASN    |      Prefix      |  IPFire Prefix  | IPFire ASN |
> +----------+------------------+-----------------+------------+
> | AS64425  | 45.148.122.0/24  | 45.148.120.0/22 | None       |
> | AS197540 | 94.16.116.0/22   | 94.16.96.0/19   | None       |
> | AS36352  | 198.23.234.0/23  | 198.23.128.0/17 | None       |
> | AS63949  | 172.105.192.0/19 | 172.104.0.0/15  | None       |
> +----------+------------------+-----------------+------------+
> 
> Thanks so much for all your hard work,
> 
> --
> Jordan